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Abstract 


Defence  Research  and  Development  Canada  (DRDC),  a  special  operating  agency  of  the 
Department  of  National  Defence,  contracted  Dalhousie  University  to  support  the  development  of 
a  management  accountability  framework  (MAP)  as  part  of  DRDC’s  EXPEDITION  09 
organizational  change  project.  The  DRDC  MAE  used  the  Treasury  Board  Secretariat  MAE  as  a 
model,  with  Expectations,  Indicators  and  Measures  tiers.  In  Cycle  2  of  the  contract,  full  Indicator 
and  Measures  tiers  were  developed,  and  the  Expectations  tier  developed  in  Cycle  1  was  refined. 
In  Cycle  2,  two  pilot  studies  were  conducted  using  web-based  surveys  of  the  DRDC  management 
cadre.  The  first  pilot  of  Cycle  2  evaluated  the  Risk  Management  and  Accountability  elements. 
The  second  pilot  of  Cycle  2  evaluated  Policy  and  Programs  and  Service  elements,  and  re¬ 
evaluated  the  Stewardship  element  from  Cycle  1.  The  results  of  the  pilot  studies  confirm  the 
feasibility  of  the  survey-based  approach  and  provide  a  partial  first  assessment  of  DRDC’s 
organizational  performance  against  indicators  for  the  elements  of  the  proposed  DRDC  MAE. 


Resume 


Recherche  et  developpement  pour  la  defense  Canada  (RDDC),  organisme  de  service  special  du 
ministere  de  la  Defense,  a  embauche  I’universite  Dalhousie  pour  E aider  a  elaborer  un  cadre  de 
responsabilisation  de  gestion  (CRG)  pour  EXPEDITION  09,  projet  de  changement 
organisationnel.  Ee  CRG  de  RDDC  s'inspire  du  CRG  du  Conseil  du  Tresor  comme  modele  et  de 
ses  paliers  Attentes,  Indicateurs  et  Mesures.  Le  cycle  2  du  contrat  a  permis  une  elaboration 
complete  des  paliers  Indicateurs  et  Mesures  et  un  peaufmage  du  palier  Attentes,  elabore  pendant 
le  cycle  1 .  Ee  cycle  2  a  ete  marque  par  la  realisation  de  deux  etudes  pilotes  par  voie  de  sondage 
sur  Internet  aupres  des  gestionnaires  de  RDDC.  La  premiere  etude  pilote  du  cycle  2  a  evalue  les 
elements  Gestion  des  risques  et  Responsabilisation.  La  deuxieme  etude  pilote  du  cycle  2  a  evalue 
les  elements  Politique  et  programmes,  et  Service,  puis  reevalue  Eelement  Gerance  du  cycle  1.  Le 
resultat  des  etudes  pilotes  a  confirme  la  faisabilite  de  Eapproche  par  sondage  et  foumi  une 
premiere  evaluation  partielle  du  rendement  organisationnel  de  RDDC  au  vu  des  indicateurs  pour 
les  elements  du  CRG  de  RDDC  propose. 
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Executive  summary 


A  DRDC  Management  Accountability  Framework:  Results  of 
Cycle  2 

Judy  A.  Baroni;  Warren  C.E.  Nethercote;  DRDC  Atlantic  CR  2009-135;  Defence 
R&D  Canada  -  Atlantic;  September  2009. 

Introduction:  Defence  Research  and  Development  Canada  (DRDC),  a  special  operating  agency 
of  the  Department  of  National  Defence,  contracted  Dalhousie  University  to  support  the 
development  of  a  management  accountability  framework  (MAP)  as  part  of  DRDC’s 
EXPEDITION  09  project.  The  DRDC  MAE  used  the  Treasury  Board  Secretariat  MAE  as  a 
model,  with  Expectations,  Indicators  and  Measures  tiers;  however,  the  focal  point  for  the  DRDC 
MAP  is  the  DRDC  management  cadre,  rather  than  the  Deputy  Minister  for  the  Treasury  Board 
Secretariat  MAP.  The  elements  of  the  DRDC  MAP  correspond  to  those  of  the  Treasury  Board 
Secretariat  MAP,  but  reflect  the  vernacular  of  the  DRDC  management  cadre. 

Results:  Under  Cycle  2  of  the  contract,  full  Indicators  and  Measures  tiers  were  developed,  and 
the  Expectations  tier,  developed  under  Cycle  1 ,  was  refined.  The  Expectations  tier  of  the  DRDC 
MAP  was  first  derived  through  consultation  with  the  DDG  and  Corporate  Services  communities, 
and  approved  in  principle  by  the  R&D  Executive  Committee. 

Subsequently,  various  modifications  were  made  to  the  elements  as  a  result  of  lessons  learned 
during  the  pilot  studies.  In  all  cases,  amendments  to  the  Expectations  tier  served  to  clarify  the 
elements  rather  than  to  introduce  substantve  changes  in  principle  or  intent. 

The  Indicators  and  Measures  tiers  of  the  DRDC  MAP  were  developed  based  on  academic 
research  of  best  practices  and  theoretical  frameworks;  utilizing  this  methodology  strengthens  the 
validity  of  the  individual  indicators  and  measures.  Assessment  criteria  of  Treasury  Board 
Secretariat  MAP  Rounds  1-VI  were  also  utilized  in  Indicators  and  Measures  development,  and 
efforts  were  made  to  align  the  Indicators  and  Measures  statements  with  those  of  the  Treasury 
Board  Secretariat  MAP  where  appropriate. 
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As  a  part  of  the  MAF  development,  two  pilot  studies  were  conducted,  each  utilizing  a  web-based 
survey  instrument  to  examine  application  of  the  MAF  elements  to  the  DRDC  management  cadre. 
The  first  pilot  of  Cycle  2  evaluated  the  Risk  Management  and  Accountability  elements;  the 
second  pilot  of  Cycle  2  evaluated  Policy  and  Programs  and  Service  elements,  and  re-evaluated  the 
Stewardship  element  from  Cycle  1 . 

Survey  questions  were  developed  based  on  DRDC  MAF  Expectations,  Indicators,  and  Measures 
tiers  for  the  target  elements,  as  well  as  input  from  pre-existing,  publicly  available  survey 
instruments.  Questions  for  the  Stewardship  element  focused  on  areas  of  concern  from  the  Cycle  1 
survey  analysis. 

In  each  survey,  the  majority  of  questions  were  closed  so  that  respondents  could  choose  only  from 
pre-determined  alternatives.  Where  appropriate,  respondents  were  given  the  option  to  write  in 
alternate  responses  (open  questions).  Question  sets  were  sorted  into  a  logic  tree  structure  (so- 
called  ‘skip  patterns’  in  survey  vernacular)  to  ensure  each  respondent  was  only  asked  questions 
applicable  to  his/her  job  level  and  duties.  Surveys  were  reviewed  by  subject  matter  experts  for 
validity  and  tested  by  a  small  subset  of  the  target  population. 

Each  survey  was  implemented  bilingually  over  the  DRDC  Intranet  through  a  secure  server  during 
separate  two  week  periods.  Survey  participants  were  provided  with  a  link  that  could  be  accessed 
from  anywhere  within  the  DRENet.  The  surveys  were  open  to  Level  2  and  Level  3  managers,  and 
Level  4  functional  authorities  who  are  eligible  to  attend  the  annual  DRDC  Managers’  Workshop. 
Response  rates  for  Surveys  2  and  3  were  53%  and  47.6%,  respectively. 

Significance:  The  second  and  third  operational  pilots  for  the  DRDC  MAF  re-confirmed  the 
feasibility  of  the  use  of  a  survey  tool  as  a  MAF  instrument.  The  results  of  the  pilots  provide  a 
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useful  partial  assessment  of  DRDC  managers’  performance  against  indicators  for  the  MAP,  and  a 
starting  point  for  the  first  operational  cycle.  It  is  believed  that  these  results  support  the  vision  of  a 
steady  state  DRDC  MAP: 

The  DRDC  Management  Accountability  Framework  (MAF)  is  a  governance  tool  that  facilitates 
dialogue  between  managers  and  executive  leadership,  helping  to  clarify  managers  ’ 
understanding  of  their  roles  and  responsibilities  and  providing  a  framework  for  assessment  and 
improvement  of  business  processes. 

Future  plans:  The  following  recommendations  arise  from  this  report: 

1 .  Defence  R&D  Canada  should  adopt  the  Management  Accountability  Pramework  proposed  in 
this  report,  together  with  the  proposed  OPls  and  proposed  annual  cycle; 

2.  Defence  R&D  Canada  should  implement  the  findings  of  the  EXPEDITION  09  operational 
pilots  for  the  proposed  DRDC  MAP,  to  provide  a  baseline  for  the  first  operational  cycle; 

3.  Implemented  recommendations  of  other  WBEs  of  EXPEDITION  09  should  be  integrated 
with  the  DRDC  MAP,  and  included  in  future  surveys  to  determine  the  success  of  their 
adoption  into  the  organization. 
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Un  cadre  de  responsabilisation  de  gestion  pour  RDDC  : 

Resultats  du  cycle  2 

Judy  A.  Baroni;  Warren  C.E.  Nethercote;  DRDC  Atlantic  CR  2009-135;  R  &  D 
pour  la  defense  Canada  -  Atlantique;  Septembre  2009. 

Introduction  :  Recherche  et  developpement  pour  la  defense  Canada  (RDDC),  organisme  de 
service  special  du  ministere  de  la  Defense,  a  embauche  I’universite  Dalhousie  pour  I’aider  a 
elaborer  un  cadre  de  responsabilisation  de  gestion  (CRG)  s’inscrivant  dans  le  projet 
EXPEDITION  09.  Comme  le  CRG  du  SCT,  le  CRG  de  RDDC  comporte  des  paliers  Attentes, 
Indicateurs  et  Mesures;  mais  sa  cible  principale  est  les  gestionnaires  de  RDDC,  pas  le  sous- 
ministre.  Ees  elements  du  CRG  de  RDDC  correspondent  a  ceux  du  CRG  du  SCT,  mais  refletent 
les  termes  utilises  par  les  gestionnaires  de  RDDC. 

Resultats  :  Le  cycle  2  du  contrat  a  permis  une  elaboration  complete  des  paliers  des  indicateurs  et 
des  mesures  et  un  peaufinage  du  palier  Attentes  elabore  pendant  le  cycle  1 .  Le  palier  Attentes  du 
CRG  de  RDDC,  elabore  apres  consultation  des  DGA  et  des  gestionnaires  des  Services 
d’entreprise,  a  obtenu  Tapprobation  de  principe  du  conseil  executif  pour  RDDC.  Par  la  suite,  ont 
ete  effectues  divers  changements  aux  elements,  en  fonction  des  lefons  tirees  des  etudes  pilotes. 
Dans  tons  les  cas,  les  changements  au  palier  Attentes  visaient  a  clarifier  les  elements  plutot  qu’a  y 
apporter  des  changements  de  fond  quant  au  principe  ou  a  Tintention. 

Les  paliers  Indicateurs  et  Mesures  du  CRG  de  RDDC  propose  ont  ete  elabores  a  partir  de 
plusieurs  elements  :  la  recherche  universitaire  sur  les  pratiques  exemplaires  et  les  cadres 
theoriques  (methodologie  qui  renforce  la  validite  des  indicateurs  et  mesures  individuels);  et  les 
criteres  d’evaluation  des  rondes  1  a  VI  du  CRG  du  SCT  (des  efforts  etant  faits,  chaque  fois 
qu’approprie,  pour  aligner  les  enonces  d’indicateurs  et  de  mesures  avec  ceux  du  CRG  du  Conseil 
du  Tresor). 
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Gouvernance 
et  orientation 
strategique 

La  charle  de 
RDDC.Ia 
St  rat  eg  ie  de 
science  et  de 
techno  logle  de 
la  Defense  et  Ie 
procedd  de 
planification 
d'activites  du 
MDN  assurent 
un  soutien 
efficace  et  one 
autoritd 
fonctionnelle 
dans  Ie 
domaine  de  la 
science  et  des 
technologies, 
pourle 

gouvernement, 
les  Forces 
canadiennes, 
la  Defense  el 
les  partenaires 
pour  la  sdcuritd 


Valeurs  de  Recherche  et  developpement  pour  la  defense  Canada 

Parses  actions,  la  gestionde  RDDC  renforce  continuellement  I'importance  des  valeurs  de 
I'agence  et  de  I'ethique  de  la  ddfense,  en  assurant  desrdsultats  aux  Forces  canadiennes,  ala 
Defense  et  aux  partenaires  pour  la  securite 


Politique  el  piogiainnies 

RODC  developpe  elmaintiert 
une  capacite  de  recherche  et 
d’ana^sequlluipermetde 
foumlrau  MDN,  aux  FC,  a 
Securite  publique  Canada  el 
aux  partenaires  pour  la  securite 
desoptionsde  politique,  une 
conception  de  programme  et 
desconseilsde  grande  qua ite. 


Employes 

RODC  a  les  employes, 
I'erwironnement  de  travail,  le 
leadership  etia  capacite 
vouluspourfoumirce  qui  se 
faitde  mieux  en  science  el 
entechnologlepourla 
defense  etiasecurite, 
maintenantelaravenir. 


Lesactivitesde  S-  etT,  sont 
axdessurle  client  et  les 
paitenaires.  Les 
programmes  sont  confus 
pourrepondrealeurs 
besoins.  El  RDDC 
encourage  etge  re  defajon 
efficace  les  partenariats. 


Gestioii  (les  risqiies 

RDDC  se  prepare  a  faire 
face  aux  risques  dans  un 
environnement  de  science 
el  de  technologle,  les  gere 
ellesexplolte. 


Le  regime  de  gestionde 
RDDC  (actifs,  argent, 
employes,  services,  etc.)  esi 
inlegre  el  efficace.  Ses 
principesdebase  sontclairs 
pourloutle  personnel. 


Responsabilisation 

La  responsablllte  pour  les 
rdsullats,  clalrement 
atirlbuee,  s'aligne  avec  les 
ressources.  Lesdelegations 
correspondent  aux 
capacites. 


Gestion  de  I'apprentlssage,  de  I’Innovation  et  des  changemenis 

RDDCfavorise  la  creativite,  I'innovation  el  le  changemeni,  fait  la  promotion  de  I'apprentissage 
organisationnel,  prise  le  savoircolleclif  etiiredes  legonsdesa  performance 


Resultats  et 
performance 


RDDC  recueille 
les 

renseignement 
s  pertinents  sur 
les  resultats 
(internes, 
service  et 
programme)  el 
les  utilise  pour 
prendre  des 
ddcisions  au 
sein  de  RDDC. 
L'information 
diffuses  au 
ministere  et  au 
public  est 
equilibrde, 
Iransparente  et 
facile  a 
comprendre 
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Gouvernance 
et  orientation 
strategique 


•Soutien  auSM 
MDNparle  SMA 
(S&T) 

•Cadre  de  gestion 
aligne  avec  les 
resultats 
strategiques 

•L'equipede 
direction  qu'll  taut 

•Prioritdsdu 
ministere  axees 
surlesrdsultats 

•Affectation  et 
reaffectation  de 
ressources  selon 
la  performance 

•Programme  de 
gestion  Integre 

•Collaboration 

horizontale 

•Analyse  de 
I'errvironnement 
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•Conduiteexemplaire 

•Valeursde  I'organisme  refletdespar  la  reallte  etIa  culture  de  ses  centres 
•Pratiquesde  gestion  axees  sur  des  valeurs 


Politk|iie  et  proyrdiiiiiies 

■Capacity  soij(«nij6  et 

cuittirede  &onsultati«n,d‘examen  et 
de  remise  en  question 
•Politique  axee  sur  les  resultats  et 
priorites  abgnees  avec  les  priorites 
horizontales  du  QOuvememerit 
•^ualualion  sociale  des 
repercussions 

•Confiance  du  SMA(S&T).  des 
cbents  et  des  partenaires 

Employes 

Main  d'oeuure  ccmptant  le  ncmbre  de 
persormes  et  les  capacites  wouBues.toii 
en  reletant  la  composition  de  la 
pcpulation  canadienrte 
>L«  lieu  de  travail  est  un  environnement 
enccur^ant  et  respectueux .  ou  sont 
respectees  les  exigences  en  mati^re  de 
langues  cficieles 
•Tous  les  riiveaux  de  gestion  font 
preuve  d'un  leadership  effectif 
•Les  employis  s'intmssant  a  leur 
travail,  ouils  out  I'occasion  d'apprertdre 
et  de  se  deveiopper 

Service 

•Qualite  de  service  suhee  et 
constamment  amelioree 

•Conseils  de  pcinte  sur  1' exploitation 
de  la  teohnol^ie 

•Impiication  proactive  des  clients  et 
partenaires 

•Rapports  effectifs  avec  cfients  et 
partenaires 

Gestion  desiisqiies 

•ftisques  geres  dans  ie  contexte  du 
profil  de  risque  du  MDN 
•Oesiion  des  risques  integr^  aux 
structures  de  planiiication  «t 
d'exploiiation  a  tous  les  ndveaux  de 
Torganisme 

-Bdstence  tfune  capacite  appuyam 
une  culture  de  gestion  des  risques 

Gerdiice 

•Systemes  de  gestion  foumissant  de 
I'informaiion  appropri^  et  une 
r^roacTion  efficace  surles  ressources, 
les  resuitais  et  les  mesures  de 
controle 

•Gestion  de  proJet  effective,  efficace  et 
economique 

‘Fonction  de  verification  interne  ^atlie 
•Respect  des  politiques ,  reglements  et 
fois 

Re$|ion$nl>IIIS(itjon 

•Reddition  de  ccmpte  et 
responsabilites  Claires  pour 
procedure  equitable  et  resultats 
•Delegation  de  pouvoir  conforme  aux 
capacites 

•Objeclifs  des  subordonnes  refietant 
ceux  fixes  a  leurs  supcrieurs 

Gestion  de  I’appranlicsage,  de  I'innovation  et  des  changements 
•RDDC  est  un  organismed'etudesdoted'une  vision  strategique 

•La  culture  del'organisine  devan ce  le  ch ang ement,  appreciellnnovation  et  soul! entlatranstormati on 
•RDDC  examine  son  propre  rendement  atn  d'ldentiter toutbesoln  de  changement 
•Oesd^egationsresponsaixlsentlesgestionnaires 

•Lesavoirel  lamemoire  dePorganisme  sont  engrangesetgerescomme  ressources  strategiques 


Resultats  et 
performance 


•Suivlet  rapports 
pourles 

programmes,  les 
services  et  les 
resultats  internes 

•Renseignements 
surle  rendement 
financleretnon 
flnanclerlntegres 
etutilisesdansla 
prise  de  decision 

•Rapports 

appuyespardes 

resultats 

mesurables 

•Criteresde 
comparalsonavec 
ce  qui  se  faitde 
mieux 

•Communications 

transparentes, 

opportuneset 

accessiblesavec 

clients,  partenaires 

et  parties 

prenantes 
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Vaieurs  de  Recherche  et  developpement  pour  la  defense  Canada 

■Adhesion  du  personnei  aux  valeursde  RDDC 

■Prise  de  decision  ethique  appuyee  pane  Programme  d'elhlque  de  la  Defense 
■Performance  de  I'organisme  mluee  selon  le  Programme  d'elhique  de  la  Defense 


Resuitats  et 
performance 


■Conflancedu 
SMA  tS&T) 
■Priorites 
coherentesen 
matiere  de 
politiques 
■Cadre  degestlon 
solide 

■Le cadre de 

gestlonappule 

I'etablissementde 

priorites,  la 

reaffectation  et 

I'alignementavec 

lesprioritesde 

I'ensembledu 

gouvernemeni 

■Lesprloritesen 

matiere  de 

changement 

concernenttous 

lesdomaines 

fonctionnels 

■Participation  de 

I'organisme  aux 

Initiatives 

horizontalestant 

admiristratives 

quesclentiflques 


Politique  et  pioyr<iimiie$ 

•Qij3lft«  des  et  des 

pnagrammes 

■Plan  d«  9«stion  des  r«ss»urc*s 
humaines  pcur  le  secteurdes 
politiques 

•In'westissemefil  dans  la  capacite  «n 
politiques  et  dans  les  outils  de 
recherche 


•Plan  stiateqlque  des  ressources 
humaines  ^abM 
•Projjres  au  vu  des  cibles  de 
ressources humairtes.dorft  la  qualite 
de  la  Mte  au  trauaM ,  les  langues 
offclelles  et  I’^uite  pour  ks  employes 
•Progres  dans  la  mesure  de 
I'implioation  des  employes  et  son 
amelioration. 

•Qualite  du  leadership 
•Quake  des  relations  de  travail 


•L'ameKoration  du  service  et  la 
planifkation  de  la  transtormatlon  sent 
en  cours 

■La  satisftetion  des  clients  et 
partenaires  se  mesure  annuellernent 
au  vu  de  erteres 

•La  collaboiation  est  essentielk  a  la 
prestation  des  services  d«  DRODC 
■RDDC rend Tinformation  technique  et 
scleritiique  rapkkment  disponible 
pour  les  parties  prenantes 


Gestion  (l«s  lisqiies 
•Preuve  9ije  la  gestion  des  risques  est 
inte^ree  a  la  prise  de  dedaoa  a  la 
planilicaticn  et  aut  rapf  orts 
■Outils  de  gestion  des  risques,  knretbn 
et  scutien  a  cesoulilsdisponblespourle 
personnel 

■  La  i^sticn  tait  pieuve  de  leadership  en 
iTiatiere  de  gestion  des  risq^s 
•Cappmhe  de  gestion  des  risques  est 
com^erTMntaire  au  prolUde  risqje  du 
ministere  dela  Oeiense  naiiorole 
•Participation  despanks^rertvites 
extemes  a  I'etaluitoo  eta  la 
communksiion  des  risques 


•Rapports  et  dossiers  sent  opportuns  et 
exacts 

•Le  depenses  i>e  d^asserrt  pas  le 
budget 

•On  se  procure  des  ressources  et  on 
les  utilise  judicieusement  pour  meindre 
des  cbjectite 

•Les  biens  soiyi  in'ventones  et  gen^  de 
facon  strategique 

•LlnforTnation  s'ulilise  et  se  prot^  de 
b^on  adequne 

•Les  realisations  attendues  sent  en 
temps  epportun  et  effectives 


RespoiisiibiliSiTtioii 

•Les  ^eclifs  de  gestion  sent 
identifables  et  mesur^es 
•Les  suberdonnes  ent  une  idee  claire 
des  devoirs  et  des  taches 
•Les  gestionnaires  et  le  personnel  ont 
la  formation  et  Tautorne  approves 
pour  assumer  les  pouvoirs  qui  kurs 
sont  d^egues 

•Un  processus  d'examen  officiel  suit  k 
progris ,  les  delegaliorts  et  le 
reo^ment 

■L'atteirite  des  ob>jecti^  par  ks 
subordennes  appuie  ks  cbjectifs  de  la 
gestion  et  la  vision  de  Torganisme 


Gestion  de  (’apprentlMsge,  de  I'innovadon  et  des  changementi 

■Progresetinvestissementdansrapprentissage  de  I'organisme 
■Progresdanslespratiquesde  gestion desconnaissances 

-Perception  que  sefontpaiiies  prenanteset  personnel  de  I’adaptabilite.  du  changement etde 
I'innovation  au  sein  de  I'organisme 
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-Qualite  du  plan 

d'activitesetdu 

rapportannuel 

-Resuitats  des 

sondagesaupris 

du  personnel  etde 

la  clientele 

-Progresdansle 

renforcementdes 

resuitats  financiers 

etde  programme, 

alnsiquedansla 

mesure  du 

rendement 

-Suivide 

I'organisme  et 

examen  du 

rendement 

-Plans 

d'evaluationaxes 
sur  les  risques 
(revises 

reQUlierement)et 
existence  d'un 
suivi 

•Rendementauvu 

dechteres 

externes 


Dans  le  cadre  de  1’ elaboration  du  CRG,  ont  ete  effectuees  deux  etudes  pilotes,  faisant  chacune 
appel  a  un  instrument  de  sondage  en  ligne  pour  etudier  1’ application  des  elements  du  CRG  aux 
gestionnaires  de  RDDC.  La  premiere  etude  pilote  du  cycle  2  a  evalue  les  elements  Gestion  des 
risques  et  Responsabilisation;  la  deuxieme,  les  elements  Politique  et  programmes,  et  Service,  puis 
reevalue  I’element  Gerance  du  cycle  1. 

Les  questions  de  sondage  pour  les  elements  cibles  ont  ete  elaborees  d'apres  les  paliers  Attentes, 
Indicateurs  et  Mesures  du  CRG  de  RDDC,  ainsi  que  d'apres  les  donnees  d'instruments  de  sondage 
existants,  librement  disponibles.  Les  questions  posees  pour  Pelement  Gerance  s’attachaient  a  des 
points  preoccupants  reveles  par  I'analyse  du  sondage  du  cycle  1 . 

Chaque  sondage  comportait  surtout  des  questions  fermees  on  les  repondants  devaient  opter 
uniquement  pour  des  choix  predetermines.  Si  approprie,  les  repondants  avaient  la  possibilite  de 
foumir  leur  propre  reponse  differente  (question  ouverte).  Les  ensembles  de  questions 
s’organisaient  selon  une  structure  d’arbre  logique  (nommee  «  instructions  passez  a  »  dans  le 
langage  des  sondages)  afin  de  poser  a  chaque  repondant  uniquement  les  questions  s'appliquant  a 
son  niveau  de  travail  et  a  ses  responsabilites.  Les  sondages  ont  fait  I'objet  d’un  examen  par  des 
specialistes,  qui  ont  verifie  leur  validite,  et  d’un  essai  preliminaire  par  un  petit  sous-ensemble  de 
la  population  cible. 

Chaque  sondage  a  ete  effectue,  dans  les  deux  langues  officielles,  dans  I’intranet  de  RDDC,  avec 
un  serveur  securise,  pendant  deux  semaines  distinctes.  On  a  foumi  aux  participants  au  sondage  un 
lien  a  un  site  libre  d’acces  pour  quiconque  au  sein  du  DRENet.  Pouvaient  y  participer  les 
gestionnaires  de  niveau  2  et  3,  ainsi  que  les  detenteurs  d’ autorites  fonctionnelles  de  niveau  4 


DRDC  Atlantic  CR  2009-135 


IX 


autorises  a  participer  a  1’ atelier  annuel  des  gestionnaires  de  RDDC.  Le  taux  de  reponse  aux 
sondages  2  et  3  a  ete  de  53  p.  cent  et  de  47,8  p.  cent,  respectivement. 

Signification  :  Les  etudes  pilotes  operationnelles  2  et  3  effectuees  pour  le  CRG  de  RDDC  ont 
reconfirme  la  faisabilite  de  I’emploi  de  sondages  comme  instruments  du  CRG.  Les  resultats  des 
etudes  pilotes  foumissent  une  evaluation  partielle  du  rendement  des  gestionnaires  de  RDDC  au  vu 
du  CRG  et  un  point  de  depart  pour  le  premier  cycle  operationnel.  Nous  estimons  que  ces  resultats 
appuient  la  vision  d’un  CRG  de  RDDC  constant : 

Le  cadre  de  responsabilisation  de  gestion  (CRG)  de  RDDC  est  un  outil  de  gouvernance  qui 
facilite  le  dialogue  entre  les  gestionnaires  et  la  direction.  II  permet  de  clarifier  le  role  et  les 
responsabilites  des  gestionnaires  et  fournit  un  cadre  pour  revaluation  et  I ’amelioration  des 
processus  operationnels. 

Recommandations  :  Au  vu  du  present  rapport,  voici  nos  recommandations 

Recherche  et  developpement  pour  la  defense  Canada  devrait  adopter  le  cadre  de 
responsabilisation  de  la  gestion  propose  dans  le  present  rapport,  ainsi  que  les  BPR  proposes 
et  le  cycle  annuel  propose; 

Recherche  et  developpement  pour  la  defense  Canada  devrait  mettre  en  oeuvre  les  constatations 
des  projets  pilotes  operationnels  d’EXPEDlTlON  09  pour  le  CRG  de  RDDC  propose,  afm 
d’ avoir  une  base  de  reference  pour  le  premier  cycle  operationnel; 

11  serait  bon  d’integrer  au  CRG  de  RDDC  les  recommandations  d’autres  elements  de  repartition 
du  travail  d’EXPEDITlON  09  et  de  les  inclure  dans  les  sondages  a  venir  pour  determiner 
leur  niveau  d’ adoption  par  Porganisme. 


X 


DRDC  Atlantic  CR  2009-135 


Table  of  contents 


Abstract . i 

Resume . i 

Executive  summary . iii 

Sommaire . vii 

Table  of  contents . xi 

List  of  figures . xiv 

List  of  tables . xvi 

Acknowledgements . xviii 

1  ....Introduction . 1 

1 . 1  Expedition  09 . 1 

1 .2  Scope  of  this  Report . 2 

1.3  Structure  of  this  Report . 2 

2....  Background . 4 

2. 1  Treasury  Board  Management  Accountability  Eramework  (TB  MAE) . 4 

2.1.1  Expectations  Tier . 7 

2.1.2  Indicators  Tier . 8 

2.1.3  Measures  Tier . 9 

2.2  Management,  Resources,  and  Results  Structures  (MRRS)  Policy . 10 

2.3  Issues  with  the  TB  MAE . 12 

3  ....Principles  and  Structure  of  the  DRDC  MAE . 15 

3.1  Principles  of  the  DRDC  MAE . 15 

3.2  Structure  of  the  DRDC  MAE . 18 

3.2.1  The  DRDC  MAE . 18 

3.2.2  Vision  of  a  Steady-State  DRDC  MAE . 20 

3.2.2. 1  The  Process . 20 

3. 2. 2.2  The  Participants . 20 

3. 2. 2. 3  The  Resources . 20 

3.2.3  Annual  DRDC  MAE  Cycle . 21 

4  ....The  Pillar  Elements . 23 

4. 1  Defence  R&D  Canada  Agency  Values . 23 

4.1.1  DRDC  Agency  Values:  Expectations . 25 

4. 1 .2  DRDC  Agency  Values:  Indicators . 25 

4. 1 .3  DRDC  Agency  Values:  Measures . 26 

4.2  Governance  and  Strategic  Direction . 29 

4.2.1  Governance  and  Strategic  Direction:  Expectations . 29 

4.2.2  Governance  and  Strategic  Direction:  Indicators . 29 

DRDC  Atlantic  CR  2009-135  xi 


4.2.3  Governance  and  Strategic  Direction:  Measures . 30 

4.3  Results  and  Performance . 34 

4.3.1  Results  and  Performance:  Expectations . 34 

4.3.2  Results  and  Performance:  Indicators . 34 

4.3.3  Results  and  Performance:  Measures . 35 

4.4  Learning,  Innovation  and  Change  Management . 38 

4.4.1  Learning,  Innovation  and  Change  Management:  Expectations . 38 

4.4.2  Learning,  Innovation  and  Change  Management:  Indicators . 39 

4.4.3  Learning,  Innovation  and  Change  Management:  Measures . 40 

5  ....The  Core  Elements . 43 

5.1  Policy  and  Programs . 43 

5.1.1  Policy  and  Programs:  Expectations . 50 

5.1.2  Policy  and  Programs  :  Indicators . 51 

5.1.3  Policy  and  Programs:  Measures . 52 

5.2  People . 55 

5.2.1  People:  Expectations . 58 

5.2.2  People:  Indicators . 58 

5.2.3  People:  Measures . 61 

5.3  Service . 64 

5.3.1  Service:  Expectations . 68 

5.3.2  Service:  Indicators . 69 

5.3.3  Service:  Measures . 71 

5.4  Risk  Management . 74 

5.4.1  Risk  Management:  Expectations . 78 

5.4.2  Risk  Management:  Indicators . 78 

5.4.3  Risk  Management:  Measures . 79 

5.5  Stewardship . 83 

5.5. 1  Stewardship:  Expectations . 84 

5.5.2  Stewardship:  Indicators . 85 

5.5.3  Stewardship:  Measures . 85 

5.6  Accountability . 88 

5.6.1  Accountability:  Expectations . 93 

5.6.2  Accountability:  Indicators . 93 

5.6.3  Accountability:  Measures . 94 

6 .. ..  Survey  Instrument  Design . 97 

6. 1  Cycle  2  Survey  Design . 97 

6.1.1  Pilot  Survey  2  Design . 98 

6. 1 .2  Pilot  Survey  3  Design . 99 

6.1.3  Assessing  Measures . 100 

7.. .. 5urvey  Analysis . 101 


xii 


DRDC  Atlantic  CR  2009-135 


7. 1  Pilot  Survey  2  Results . 101 

7.2  Pilot  Survey  3  Results . 144 

8.. ..  Findings . 182 

8.1  Policy  and  Programs . 1 83 

8.2  People . 185 

8.3  Service . 187 

8.4  Risk  Management . 188 

8.5  Stewardship . 189 

8.6  Accountability . 191 

9.. ..  Recommendations . 193 

Annex  A  ..English  Surveys . 199 

A.l  Survey  2 . 199 

A. 2  Survey  3 . 217 

Annex  B...  French  Surveys . 245 

B.  1  Sondage  2 . 245 

B.2  Sondage  3 . 263 

Bibliography . 290 

List  of  abbreviations  and  acronyms . 300 

Distribution  list . 303 


xiii 


DRDC  Atlantic  CR  2009-135 


List  of  figures 


Figure  1:  Expedition  09  Project  Structure . 1 

Figure  2:  TB  MAF,  Expectations  Tier . 7 

Figure  3:  TB  MAF,  Indicators  Tier . 8 

Figure  4:  TB  MAF,  Measures  Tier . 9 

Figure  5:  PAA  and  MAF  Interaction . 1 1 

Figure  6:  DRDC  MAF,  Expectations  Tier . 18 

Figure  7:  DRDC  MAF,  Indicators  Tier . 19 

Figure  8:  DRDC  MAF,  Measures  Tier . 19 

Figure  9  :  DRDC  Agency  Values,  Expectations  Tier . 25 

Figure  10  :  DRDC  Agency  Values,  Indicators  Tier . 26 

Figure  1 1  :  DRDC  Agency  Values,  Measures  Tier . 27 

Figure  12:  Governance  and  Strategic  Direction,  Expectations  Tier . 29 

Figure  13:  Governance  and  Strategic  Direction,  Indicators  Tier . 30 

Figure  14:  Governance  and  Strategic  Direction,  Measures  Tier . 31 

Figure  15:  Results  and  Performance,  Expectations  Tier . 34 

Figure  16:  Results  and  Performance,  Indicators  Tier . 35 

Figure  17:  Results  and  Performance,  Measures  Tier . 36 

Figure  18:  Learning,  Innovation  and  Change  Management,  Expectations  Tier . 39 

Figure  19:  Learning,  Innovation  and  Change  Management,  Indicators  Tier . 40 

Figure  20:  Learning,  Innovation  and  Change  Management,  Measures  Tier . 41 

Figure  21:  Canada’s  Science  and  Technology  Framework . 44 

Figure  22:  Easton’s  System  Framework:  A  Public  Policy  Model . 47 

Figure  23:  Policy  and  Programs,  Expectations  Tier . 51 

Figure  24:  Policy  and  Programs,  Indicators  Tier . 52 

Figure  25:  Policy  and  Programs,  Measures  Tier . 53 

Figure  26:  People,  Expectations  Tier . 58 

Figure  27:  People,  Indicators  Tier . 60 

Figure  28:  People,  Measures  Tier . 61 

Figure  29:  Service  Equation . 64 

Figure  30:  Quality  Gap  Analysis  Model . 68 


xiv  DRDC  Atlantic  CR  2009-135 


Figure  31:  Service,  Expectations  Tier . 69 

Figure  32:  Service,  Indicators  Tier . 70 

Figure  33:  Service,  Measures  Tier . 72 

Figure  34:  Risk  Management,  Expectations  Tier . 78 

Figure  35:  Risk  Management,  Indicators  Tier . 79 

Figure  36:  Risk  Management,  Measures  Tier . 81 

Figure  37:  Stewardship,  Expecations  Tier . 84 

Figure  38:  Stewardship,  Indicators  Tier . 85 

Figure  39:  Stewardship,  Measures  Tier . 86 

Figure  40:  Jacques'  Three-level  Managerial  Einkage . 91 

Figure  41:  Accountability,  Expectations  Tier . 93 

Figure  42:  Accountability,  Indicators  Tier . 94 

Figure  43:  Accountability,  Measures  Tier . 95 


DRDC  Atlantic  CR  2009-135  XV 


List  of  tables 


Table  1 :  TB  MAP  Elements . 5 

Table  2:  Colours  of  the  MAP . 6 

Table  3:  MAP  Problem  Areas  and  Possible  Mitigation  Strategies . 14 

Table  4:  S&T  Strategy  Areas  and  Deliverables . 17 

Table  5:  Annual  MAP  Cycle . 22 

Table  6:  Values  and  Ethics  in  DRDC  and  the  Public  Service . 24 

Table  7:  Evidence  and  Instruments  for  Defence  R&D  Canada  Agency  Values . 28 

Table  8:  Evidence  and  Instruments  for  Governance  and  Strategic  Direction . 32 

Table  9:  Evidence  and  Instruments  for  Results  and  Performance . 36 

Table  10:  Evidence  and  Instruments  for  Eearning,  Innovation  and  Change  Management . 41 

Table  1 1 :  Alignment  of  S&T  Strategy  with  Portfolio  Management . 50 

Table  12:  Evidence  and  Instruments  for  Policy  and  Programs . 54 

Table  13:  Evidence  and  Instruments  for  People . 62 

Table  14:  Evidence  and  Instruments  for  Service . 72 

Table  15:  Comparison  of  IRM  Key  Elements  against  DND/CP  Practice . 75 

Table  16:  Alignment  of  High  Resiliency  Organization  Theory  with  TBCS  Integrated  Risk 
Management  Pramework  and  DND  Integrated  Risk  Management  Policy  and 
Guidelines . 77 

Table  17:  Risk  Management  in  the  Treasury  Board  MAP,  Round  VI . 80 

Table  18:  Evidence  and  Instruments  for  Risk  Management . 81 

Table  19:  Evidence  and  Instruments  for  Stewardship . 86 

Table  20:  Accountability  Systems . 89 

Table  21:  Evidence  and  Instruments  for  Accountability . 95 

Table  22:  DRDC  Management  Eevels . 97 

Table  23:  Ordinal  Relationship  Assessment  Scale . 100 

Table  24:  MAP  Assessment  Scale  -  Round  IV . 183 

Table  25:  DRDC  MAP  Assessment,  Policy  and  Programs . 185 

Table  26:  DRDC  MAP  Assessment,  People . 186 

Table  27:  DRDC  MAP  Assessment,  Service . 188 

Table  28:  DRDC  MAP  Assessment,  Risk  Management . 189 

Table  29:  DRDC  MAP  Assessment,  Stewardship . 191 


xvi  DRDC  Atlantic  CR  2009- 1 3  5 


Table  30:  DRDC  MAP  Assessment,  Accountability . 192 

DRDC  Atlantic  CR  2009-135  xvii 


Acknowledgements 


The  authors  would  like  to  acknowledge  the  significant  contributions  of  a  number  of  colleagues  or 
clients. 

Dr.  Dominika  Wranik,  Dr.  Fazley  Siddiq,  and  Prof  Michael  Davies  of  the  Dalhousie  University 
School  of  Public  Administration  provided  advice  on  the  design  and  implementation  of  surveys. 
Ms.  Cathy  Poulain,  of  DRDC  Atlantic,  implemented  the  surveys  on  the  DRDC  Atlantic 
ORACLE®  server,  and  supported  the  execution  of  the  surveys.  Messrs.  Ed  Pitula,  Jim  S. 
Kennedy,  Keith  Hendy,  Dan  Hutt,  and  David  Oxford,  together  with  Drs.  Jim  L.  Kennedy,  Joe 
Arbour  and  Calvin  Hyatt,  all  of  DRDC,  willingly  performed  pilot  testing  of  survey  instruments. 

Drs.  Peter  Aucoin  and  Kevin  Quigley  of  the  Dalhousie  University  School  of  Public 
Administration  provided  subject  matter  expertise  in  the  development  of  the  DRDC  MAE  and 
critical  review  of  the  analyses  of  the  pilot  studies.  Dr.  Evert  Eindquist  of  the  University  of 
Victoria  graciously  provided  an  advance  copy  of  his  critique  of  the  Treasury  Board  MAE  for  the 
authors’  use. 

Drs.  Ross  Graham  and  Malcolm  Vant  provided  executive  oversight  on  behalf  of  DRDC’s 
Executive  Committee,  in  their  role  as  co-Eeaders  of  EXPEDITION  09  WBE  1.4. 


xviii 


DRDC  Atlantic  CR  2009-135 


1  Introduction 


This  report  is  one  of  three  in  a  series  of  reports  chronicling  the  development  of  a  Treasury  Board 
style  Management  Accountability  Framework  for  Defence  Research  Development  Canada.  This 
report  provides  a  knowledge  base  for  subject  matter  experts  to  aid  in  the  implementation  of  a  DRDC 
MAF.  For  an  executive  overview  of  the  DRDC  MAF  project  in  its  entirety,  please  see  Baroni  and 
Nethercote,  A  DRDC  Management  Accountability  Framework:  Final  Report,  DRDC  Atlantic  CR 
2009-136. 

1.1  Expedition  09 

Defence  Research  and  Development  Canada  (DRDC),  a  special  operating  agency  of  the  Department 
of  National  Defence,  contracted  Dalhousie  University  to  support  the  development  of  a  management 
accountability  framework  (MAF)  as  part  of  DRDC’s  EXPEDITION  09  project. 

EXPEDITION  09  is  DRDC’s  core  organizational  change  initiative,  following  an  earlier  change 
initiative,  EXPEDITION  07.  The  EXPEDITION  series  established  the  principle  of  an  ‘umbrella,’ 
agency-wide  change  program,  under  the  control  of  the  DRDC  Executive  Committee,  demonstrating 
organizational  support.  Figure  1  illustrates  the  project  structure  of  EXPEDITION  09  schematically. 
This  contract  report  supports  Task  3  ofWBE  1.4:  Management  Eeadership  Capabilities. 


WBEi 


EXPEDITION  09 


WBE  1.4: 

Management  Leadership  Capabilities 


Task  1:  Management  Workload 


Task  2:  Management  Leadership 
Development 


Task  3:  Management  Accountability 
Framework 


WBEn 


Figure  F  Expedition  09  Project  Structure 
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Under  the  contract,  Dalhousie  University  supported  a  cyclic  development  process  for  a  DRDC 

management  accountability  framework  (MAP),  as  follows: 

Cycle  ‘zero’,  which  established  executive  direction  and  developed  a  basic  MAP; 

Cycle  1,  which  developed  an  Expectations  Tier  based  on  the  TB  MAP  through  consultation  with 
DRDC  managers;  developed  Indicators  and  Measures  for  Stewardship  and  Accountability,  and 
conducted  an  operational  pilot  of  the  Stewardship  element  through  a  survey  of  managers;  and. 

Cycle  2,  which  completed  the  development  of  all  elements  of  the  DRDC  MAP  and  completed 
survey-based  operational  pilots  of  all  the  core  elements. 

1 .2  Scope  of  this  Report 

This  report  describes  the  results  of  the  following  activities  under  Cycle  2  of  the  process: 

Develop  a  Cycle  2  MAP  based  on  the  Cycle  ‘zero’  ‘straw-man  MAP’*,  the  results  of  the  Cycle  1 
operational  pilot,  through  academic  research,  with  reference  to  appropriate  TBS  documents 
and  policies,  and  in  consultation  with  DRDC  Deputy  Directors  General,  DRDC  Corporate 
Services,  functional  authorities  and  managers. 

♦  The  Cycle  2  MAP  includes  complete  Expectations,  Indicators,  and  Measures  tiers; 

Plan,  execute  and  analyze  the  second  operational  pilot  survey  of  the  Risk  Management  and 
Accountability  elements; 

Plan,  execute  and  analyze  a  third  operational  pilot  survey  of  the  Policy  and  Program  and  Service 
elements; 

Revisit  the  Stewardship  element  as  a  part  of  the  third  survey  pilot; 

Working  in  parallel  with  these  operational  pilots,  develop  Indicators  and  Measures  for  the 
remaining  elements; 

Revise  the  Cycle  2  MAP  as  necessary  as  a  result  of  research  and  survey  results  to  achieve  a  final 
DRDC  MAP. 

1 .3  Structure  of  this  Report 

The  structure  of  the  report  is  as  follows: 

Section  2  introduces  the  Treasury  Board  MAP,  providing  background  on  its  development  and 
implementation;  it  also  references  the  Management,  Resources,  and  Results  Structure  (MRRS); 

Section  3  outlines  the  principles  of  a  DRDC  MAP,  referencing  the  DRDC  corporate  mandate, 
strategy,  vision  and  mission.  This  section  also  proposes  an  annual  MAP  cycle  schedule  as  well 
as  the  anticipated  process,  resources,  participants  necessary  to  adopt  the  proposed  MAP  cycle; 

Section  4  introduces  the  four  Pillar  Elements  of  the  DRDC  MAP; 

Section  5  details  the  development  of  the  six  Core  Elements  of  the  MAP: 


*  Warren  Nethercote,  A  DRDC  Management  Accountability  Framework:  A  Straw-man,  April  2008,  Dalhousie  University 
School  of  Public  Administration,  Discussion  Paper,  April  21,  2008. 
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♦  Policy  and  Programs 

♦  People 

♦  Service 

♦  Risk  Management 

♦  Stewardship,  and 

♦  Accountability; 

Section  6  describes  the  design  of  the  survey  instruments  for  operational  pilots  2  and  3,  including 
survey  implementation  and  limitations; 

Section  7  provides  a  detailed  analysis  of  the  results  of  the  two  operational  pilot  surveys; 

Section  8  explains  and  categorizes  the  findings  of  the  two  operational  pilot  surveys  according  to 
the  six  Core  Elements  of  the  DRDC  MAE.  This  section  also  rates  organizational  performance 
according  to  the  element  indicators  assessed  by  our  surveys; 

Section  9  concludes  Cycle  2  with  recommendations  for  DRDC  based  on  the  results  of  academic 
research  and  the  findings  of  our  operational  pilots; 

Annexes  A  and  B  provide  the  text  of  the  English  and  French  language  surveys,  respectively; 

A  bibliography  lists  131  documents  and  references  utilized  by  the  authors. 
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2  Background^ 


2.1  Treasury  Board  Management  Accountability  Framework  (TB 
MAF)^ 

Since  1 997,  the  Treasury  Board  and  its  Secretariat  have  been  the  designated  management  board  for 
the  Government  of  Canada,  with  a  variety  of  roles  including  “leading  and  providing  expertise  in  the 
development  of  an  agenda  to  improve  management  practices.”'^  This  management  improvement 
agenda  expressed  in  Results  for  Canadians  is  arguably  the  genesis  of  the  TB  Management 
Accountability  Framework^. 

One  element  of  Results  for  Canadians  was  the  Modem  Comptrollership  Initiative  (MCI),  which 
called  for  “more  effective  decision-making,  greater  accountability,  a  mature  approach  to  risk 
management,  results-based  control  systems,  and  shared  values  and  ethics.”®  Strong  and  clear 
accountability  was  a  central  element  of  the  MCI,  to  ensure  that  the  four  management  commitments 
of  the  Government  of  Canada  would  be  met:  Citizen  Focus;  Values;  Results;  and  Responsible 
Spending. 

The  Management  Accountability  Framework  was  the  successor  initiative  to  the  MCI,  subsuming 
modem  comptrollership  along  with  other  key  elements  of  modem  management  to  provide 
management  expectations  for  deputy  heads  and  public  service  managers.’ 

In  2003,  the  Treasury  Board  introduced  a  draft  MAF  with  ten  core  elements  of  public  service 
management  for  bilateral  discussions  between  deputy  ministers  and  the  Secretary  of  the  Treasury 
Board.* *  The  MAF  uses  an  integrative  approach,  combining  the  ten  elements  to  show  what  the 
Treasury  Board,  and  the  public  indirectly,  expects  of  managers.  These  ten  elements,  shown  in  Table 
1,  are  meant  to  succinctly  capture  what  Jim  Judd,  Secretary  of  the  Treasury  Board  Secretariat  at  the 
time,  called  ‘the  questions  that  keep  Deputy  Ministers’  up  at  night”.  ® 


2 

Editor’s  note:  The  rest  of  this  report  includes  numerous  internet  links.  Unless  otherwise  noted,  they  were  current  as  of  30  June  2010. 

*  This  section  draws  substantially  from  Craig  O’Blenis  and  Warren  Nethercote,  A  DRDC  Management  Accountability 
Framework:  Cycle  I  Final  Report,  DRDC  Atlantic  CR  2008  -  188,  October  2008,  pp.  5  -9. 

Treasury  Board  of  Canada  Secretariat,  Results  for  Canadians:  A  Management  Framework  for  the  Government  of 
Canada,  March  30,  2000,  p.  18. 

^  Evert  Lindquist,  “How  Ottawa  Assesses  Departmental/  Agency  Performance:  Treasury  Board’s  Management 
Accountability  Framework”  in  How  Ottawa  Spends.  2009  -  2010:  Economic  Upheaval  and  Political  Dysfunction.  Allan 
M.  Maslove,  ed,  forthcoming,  October  2009. 

®  Jean  Depuis,  Modern  Comptrollership  and  the  Management  Accountability  Framework,  Parliamentary  Information  and 
Research  Service:  PRB  06-23E,  April  3,  2006,  p.  3. 

^  Jean  Depuis,  p.  9. 

*  Treasury  Board  of  Canada  Secretariat,  MAF  Assessments  -  Round  I,  available  from:  http://www.tbs-sct.gc.ca/maf- 
crg/assessments-evaluations/assessments-evaluations  e.asp#rl . 

*  Ivan  Blake,  video  presentation  on  Management  Accountability  Framework  at  CCMD,  2005,  available  from: 
http://www.tbs-sct.gc.ca/maf-crg/documents/video-video/video-video  e.asp  . 
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Table  1:  TB  MAF  Elements 


Governance  and  Strategic  Directions 

Public  Service  Values 

Policy  and  Programs 

People 

Risk  Management 

Stewardship 

Results  and  Performance 

Teaming,  Innovation  and  Change  Management 

Citizen-focused  Service 

Accountability 

In  2004,  the  Treasury  Board,  with  35  federal  departments  and  agencies,  developed  management 
priorities  based  on  the  ten  elements. From  this,  the  Treasury  Board  identified  41  indicators  and  in 

2005,  it  assessed  53  departments  and  agencies  against  the  indicators.  At  this  time,  the  Treasury 
Board  did  not  intend  to  review  management  quality;  instead,  it  was  looking  for  organizational 
performance  compared  to  the  specific  indicators."  The  latest  round  of  assessments,  conducted  in 

2006,  expanded  to  55  departments  and  agencies."  The  Treasury  Board  revised  the  2004  indicators 
based  on  departmental  and  agency  input, "  and  to  focus  on  management  aspects  it  could  assess  easily 
through  normal  oversight  activities.  Information  sources  included  estimates,  management  resources 
and  results  structures  (e.g.,  program  activity  architectures.  Memoranda  to  Cabinet,  and  Treasury 
Board  submissions),  and  existing  consultations. 

The  three  MAF  tiers  detail  the  commitments  senior  managers  make  as  departmental  leaders.  The 
top  tier.  Expectations,  defines  leadership  standards  and  recognizes  that  successful  management 
relies  on  the  connection  between  each  element.  The  second  tier.  Indicators,  supports  Expectations, 
providing  practical  descriptions  easily  adapted  to  an  agency’s  unique  context.  The  bottom  tier. 
Measurements,  is  an  expression  of  each  element’s  performance  standards.  While  the  expectations 
are  constant,  a  MAF  recognizes  departments  and  agencies  are  unique  by  allowing  flexibility  in 
definition  of  Indicators  and  Measures. 

The  MAF  is  a  living  document  and  Indicators  and  Measures  in  particular  may  change  based  on 
changes  in  its  foundation  pieces,  such  as  a  change  in  the  Organizational  Mandate,  changes  to  the  TB 
values,  or  changes  to  legislation  affecting  core  elements.  Ivan  Blake,  then  the  Executive  Director  of 
the  Comptrollership  Modernization  Directorate  which  was  responsible  for  overseeing  the  creation 
of  the  MAF,  has  said  the  TB  MAF  is  open  to  change:  “Jim  Judd  [Secretary  of  the  Treasury  Board 
Secretariat,  May  2002  -  November  2004]  has  been  absolutely  adamant:  the  document  will  never  be 


Treasury  Board  of  Canada  Secretariat,  MAF  Assessments  -  Round  II,  available  from:  http://www.tbs-sct.gc.ca/maf- 
crg/assessments-evaluations/assessments-evaluations  e.asp#r2. 

"  Treasury  Board  of  Canada  Secretariat,  MAF  Assessments  -  Round  III,  available  from:  http://www.tbs-sct.gc.ca/maf- 
crg/assessments-evaluations/2005/index  e.asp. 

"  Treasury  Board  of  Canada  Secretariat,  List  of  Departments  and  Agencies  assessed  during  MAF  Round  IV,  available 
from:  http://www.tbs-sct.gc.ca/maf-crg/assessments-evaluations/20Q6/departments-ministeres  e.asp. 

"  Treasury  Board  of  Canada  Secretariat,  MAF  Assessments  -  Round  IV,  available  from:  http://www.tbs-sct.gc.ca/maf- 
crg/assessments-evaluations/assessments-evaluations  e.asp#r4  . 

Treasury  Board  of  Canada  Secretariat,  MAF  Elements  and  Indicators  -  2006,  available  from:  http://www.tbs- 
sct.gc.ca/maf-crg/indicators-indicateurs/2006/elements-elements  e.asp. 
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carved  in  stone”.  This  has  proven  to  be  true  to  some  degree,  as  each  year,  the  TBS  adjusts  and 
refines  the  information  sources  it  uses  to  assess  departmental  performance.'® 

The  MAT  was  designed  as  a  visual  graphic  of  a  flow  chart,  which  represents  the  relationships 
among  the  elements.  Each  tier  of  the  TB  MAE  has  ten  elements  that,  according  to  TBS,  describe  key 
characteristics  of  management  performance,  in  which  four  pillar  elements  surround  six  core 
elements.  The  authors  of  this  study  propose  that  this  layout  helps  describe  the  relationship  of  the 
elements.  The  MAE  begins  on  the  left  hand  side,  with  Governance  and  Strategic  Direction  and  ends 
on  the  right  side  with  Results  and  Performance.  The  directional  arrows  which  take  the  observer 
from  the  left  to  the  right  of  the  MAE  cross  through  the  six  core  elements,  which  are  flanked  by  the 
two  remaining  pillar  elements.  This  layout  suggests  that  the  six  core  elements,  bounded  by  Values 
and  Learning,  Innovation  and  Change  Management  principles,  are  the  means  by  which  Governance 
and  Strategic  Direction  are  translated  into  Results  and  Performance. 

The  colours  used  in  the  MAE  are  also  significant;  the  four  colours  were  chosen  to  represent  the  four 
key  areas  of  the  MAP  Vision  for  the  Public  Service,  as  shown  in  Table  2,  below,  and  further  define 
the  relationships  among  the  elements.'^ 


Table  2:  Colours  of  the  MAP 


Blue 

Is  founded  on  values  that  will  never  change  and  an 
appreeiation  that  everything  else  may 

Red 

Exists  for  the  express  purpose  of  turning  the  direetion 
of  Government  into  results  for  Canadians 

Yellow 

Continuously  improves  the  quality  of  its  analysis  and 
its  understanding  of  the  perspeetives  of  its  citizens 

Green 

Frames  all  its  deeisions  with  due  regard  to  risk, 
rigorous  stewardship  of  publie  resourees,  the  eapaeity 
and  eondition  of  its  human  resources  and  the 
systematie  management  of  its  accountabilities 

'^Ivan  Blake,  2005. 

Treasury  Board  of  Canada  Secretariat,  MAP  VI  Methodology  Changes  2008-2009,  available  from:  http://www.tbs- 
sct.gc.ca/maf-crg/indicators-indicateurs/2008/elements-elements-eng.asp  . 

”  Robert  Fonberg,  Measuring  Management:  The  Canadian  Experience,  Treasury  Board  of  Canada  Secretariat  Presentation 
Notes,  National  School  of  Government  Conference  on  21st  Century  Public  Services,  London,  United  Kingdom,  6  June 
2006;  and  Virginie  Ethier,  Management  Accountability  Framework  (MAP)  Overview,  PPX  Presentation,  May  15,  2007. 
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Expectations  Tier 


Figure  2:  TB  MAF,  Expectations  Tier 


The  elements  of  the  Expectations  tier  in  Figure  2  describe  the  Treasury  Board’s  expectations  of 
senior  managers  in  the  Government  of  Canada'*.  This  framework  overcomes  silo  thinking  by 
emphasizing  the  relationship  between  each  of  the  elements.  For  example,  although  Stewardship  can 
be  narrowly  considered  a  financial  activity,  managing  human  resources  has  significant  impact  on 
financial  performance.  In  addition  to  the  People  element.  Policy  and  Program  directives  connect 
intimately  to  Stewardship  decisions.  A  manager  may  find  that  project  failure  is  due  to  limited 
resources,  but  without  sound  financial  management,  it  is  a  challenge  to  overcome  the  problem.  A 
manager  succeeds  at  delivering  high-quality  public  service  by  meeting  the  expectations  of  each 
element.  To  meet  expectations,  managers  need  performance  indicators  and  clear  measures,  detailed 
in  the  remaining  MAF  tiers. 


**  Treasury  Board  of  Canada  Secretariat,  Management  Accountability  Framework,  2003,  available  from: 
http://www.tbs-sct.gc.ca/maf-crg/documents/booklet-livret/booklet-livret-eng.asp. 
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Indicators  Tier 


Public  Service  Values 


•  Exemplary  conduct 

•  Public  service  valuM  tailored  to  rai 

•  Value«ebs*ed  mana^ment  practio 


bes/cuiture  of  department 


Governance 

and 

Strategic 

Directions 

•  Si4)port  to 
minister.  Cabinet 
and  Parliament; 

•  Manapement 
framevirork  aligned 
to  strategic 
outcomes 

•  The  right 
executive  team 

•  Results-focussed 
corporate  priorities 

•  Strategic  resource 
alocation/ 
reallocation  based 
on  performance 

•  Integrated  agenda 
for  management 
excellence 

•  Horizontal 
colaboration 

«  Errvironmental 
scanrkng 


Learning,  Innovation  and  Change  Management 

Stratoglc  organizational  learning,  a  capM%iio  anttcipefand  adjust  to  change,  and  aiMsposition 
to  transformation 
A  culture  of  Innovation 
Performar>ce  as  a  guide  to  change 
Delegations  as  an  Instrument  of  empowerment 

Corporate  kfxiwledge  and  memory  captured  and  managed  aretrateglc  resources 


Results 

and 

Performance 

•  Corporate 
monitoring 
artd  reporting  of 
program,  service 
ar>d  internal 
results 

•  Integrated 
financial  and 
rK>n- financial 
performance 
information  used 
in  corporate 
decision  making 

•  Departmental 
reporting  based 
on  ni>easurable 
outcomes 

•  Benchmark 
against  the  best 

•  Transparent, 
timely  and 
accessible 
communications 
with  citizens 
and  Parliament 


Figure  3:  TB  MAF,  Indicators  Tier 

The  indicators,  as  seen  in  Figure  3,  describe  the  breadth  of  the  elements'^.  The  Treasury  Broad 
intends  that  Deputy  Ministers  engage  their  executive  councils  in  discussions  of  the  items  listed  in 
the  indicators  to  determine  how  well  their  department  reflects  the  MAF’s  expectations.  In  later 
rounds  of  assessment,  TBS  refers  to  Indicators  as  ‘Areas  of  Management’;  however,  the  descriptive 
material  on  the  TBS  website  continues  to  use  the  nomenclature  of  Indicators;  therefore,  we  have 
preserved  this  original  wording. 


TBS,  Management  Accountability  Framework. 
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Measures  Tier 


Governance 

and 

Strategic 

Directions 

•  Minister's 
confidence  in 
departmental 
support 

•  Perceived 
cof>erence  of 
poNcy  agenda 

•  Strength  of  the 
managemeni  team 

•  Corporate 
management 
framework  used 
for  priority  setting, 
reaHocati^  arxl 
alignment  to 
government 'Wide 
priorities 

•  Management 
improvement 
agenda 

integrating  HR, 
comptroMership, 
servicer  etc. 

•  Leadership/ 
participation  in 
PS'Wkte  initiatives 


Pubiic  Service  Values 

Customized  puhHc  service  values  statement  and  ethical  guidelines  regularly  discussed  with  all  staff 
Sound  advisory  and  recourse  mechanisms  In  place 
Orientation,  learning  and  other  tools  to  support  staff 

Staff  assessment  of  organizational  performance  against  PS  values  and  ethics 


Policy  and 
Programs 

Confirtence  of  the  minister 
and  the  PCO  in  the  quality 
of  policy  options  aivl  advice 
RecrultmenydevelopmenV 
succession  plans  for  policy 
community 
Investments  in  policy 
capacily/analylic  tools 


•  Comprehensive  HR  devetop- 
ment  plan  in  place,  including 
leadership,  recruitment, 
retention,  succession, 
learrvng,  QWL,  OU  EE 

•  Process  against  HR  targets 

•  Progress  in  nneasuring/ 
improving  employee 
engagement 

•  Quality  of  teadership 

■QuIWy  of  labour  relations 


Citizen-focussed 

Service 

►  Service  improvement  arwl 
transformation  plans  in  place 
for  major  services/ 
regulatory  programs 

^  Client  satisfaction  measured 
annually 

*  Client  satisfaction  targets 
and  results 

►  Progress  toward  GOL  targets 
» Colaboratlon  with  other 

governments  and  partners 

►  Information  for  citizens 


Risk  Management 

'  Corporate  Risk  Proftlef 
reviewed  regularly 
'  Tools,  training,  support 
for  staff 

Evidence  of  risk  considerations 
in  strategic  plannlr>g 
Engagement  of  external 
stakeholders  In  assessing/ 
communlcatir>g  risks 


Stewardship 

*  RIsk'based  audit  plans  (reviewed 
regularly)  arvj  follow-up 

*  Progress  in  Integrating 
corporate  information  systems 
andconliols 

*  Audit  findings  and  control 
glares 

*  Quality  assurance  in  contracting, 
financial,  krxrwledge  and 

asset  managemenL  and 
,  rr  atowoiwtoWp _ , 


Accountability 

>  Clarity  of  accountabilities 

I  Dategatiorts  regJarty 

reviewed 

•  Executive  committee  oversight 
of  performance  management 
and  regular  review 
of  performarrce 
■  AJignnrent  of  individual  with 
corporate  commitments 


Results 

and 

Performance 

•  Quality  of  RPPs 
and  OPRs 

•  Staff  ar>d  client 
survey  results 

•  Progress  in 

strengthening 

financial  arKi 
program  results, 
and  performarKe 
measurement 

•  Corporate 
monitoring  and 
review  of 
performance 

•  Risk-based 
evaluation  plans 
(reviewed  regularly) 
and  follow-up 

«  Performance 
against  external 
berKhmarks 


Learning,  Innovation  and  Change  Management 

■  Progress  in  improving  organizational  learning  and  knowledge  management  practices 

•  Investments  in  organizational  learning 

•  Stakehoider/stan  perceptions  of  organizational  adaptability,  change  and  innovation 

■  Performance  measurement  used  to  improve  organizational  results 


Figure  4:  TB  MAF,  Measures  Tier 

Measures  are  used  to  determine  how  a  department  functions;  unlike  Expectations  and  Indicators, 
measures  may  change  in  response  to  departmental  needs^°.  Sources  of  information  change, 
particularly  when  reporting  methods  change.  The  measures  tier  of  the  Treasury  Board  MAF,  shown 
in  Figure  4,  lists  potential  sources,  while  acknowledging  a  department  may  have  other,  accurate 
information  sources  that  describe  departmental  performance.  The  Treasury  Board  Secretariat  and 
Privy  Council  Office  use  information  found  through  the  measures  to  engage  Deputy  Ministers  on 
their  department’s  performance.  Depending  on  performance,  certain  measures  may  be  more 
important  for  discussion  than  others.  For  example,  a  department  may  perform  well  at  risk 
management  but  could  benefit  from  improvement  in  its  accountability  guidelines.  In  this  case,  the 
Treasury  Board,  Privy  Council  Office,  and  Deputy  Minister  would  focus  on  accountability  at  the 
expense  of  risk  management. 


In  later  rounds  of  assessment,  TBS  refers  to  Measures  as  ‘Fines  of  Evidence’;  however,  the 
descriptive  material  on  the  TBS  website  continues  to  use  the  nomenclature  of  Measures;  therefore, 
we  have  preserved  this  original  wording. 


TBS,  Management  Accountability  Framework. 
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2.2  Management,  Resources,  and  Results  Structures  (MRRS) 
Policy 

The  Management,  Resources,  and  Results  Structures  (MRRS),  introduced  in  2005,  is  a  policy  set  out 
by  the  TBS  which  requires  all  government  departments  “to  design  and  manage  their  programs  in  a 
manner  that  best  achieves  results  for  Canadians.”^' 

The  goal  of  the  MRRS  policy  is  to  integrate  the  reporting  of  financial  and  non-financial  program 
performance  information  to: 

support  informed  decisions  on  program  management,  and  new  program  proposals 

support  improved  allocation  and  reallocation  decisions  in  individual  departments  and  across  the 
government 

improve  accountability  reporting  to  Parliament. 

The  MRRS  has  four  interdependent  policy  components  which  work  together  to  align  programs, 
resources,  activities  and  results. When  reported  together,  these  components  seek  to  describe  results 
attained  in  terms  of  the  total  resources  required  to  achieve  them.^"*  They  are: 

Strategic  outcomes:  Departments  have  appropriate,  clearly  identified,  and  measuarable  intended 
outcomes  for  all  programs  and  activities,  which  reflect  the  mandate  and  vision  of  the 
organization. 

Program  Activity  Architecture  (PAA):  Programs  and  associated  sub-programs  are  clearly  linked. 
Planned  resources,  expected  program  results,  and  responsibilities  are  identified  at  each  level. 
All  activities  support  the  departments’  strategic  outcomes. 

Resource  and  Results  Linkages:  The  allocation  of  departmental  resources  is  clearly  connected  to 
the  achievement  of  strategic  outcomes,  and  resources  are  utilized  efficiently  to  achieve  results. 

Governance  Structure:  Departments  have  a  framework  of  decision-making  mechanisms, 
responsibilities,  and  accountabilities  by  which  individuals  and  departments  can  systematically 
report  intended  results  against  actual  results  and  the  resources  utilized. 

The  cornerstone  of  the  MRRS  is  the  Program  Activity  Architecture  (PAA)  component.  The 
Program  Activity  Architecture  interacts  with  the  Management  Accountability  Framework  in  two 
ways;  first,  by  “providing  a  standard  basis  for  reporting  to  citizens  and  Parliament  on  the  alignment 
of  resources,  program  activities  and  results”,^^  it  generates  reports  which  can  be  used  as  information 
measures  in  the  MAF  organizational  assessment. 


Treasury  Board  of  Canada  Secretariat,  Policy  on  Management,  Resources  and  Results  Structure,  available  from: 
http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?id=14252&section=text . 

TBS,  Policy  on  Management,  Resources  and  Results  Structure. 

Lee  McCormack,  Getting  the  Foundations  Right,  presentation  notes,  November  30,  2004;  TBS,  Policy  on  Management, 
Resources  and  Results  Structure. 

Lee  McCormack,  Getting  the  Foundations  Right. 

TBS,  Policy  on  Management,  Resources  and  Results  Structure. 
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Program 

Activity 


Organizational 

Management 

Performance 

Figure  5:  PAA  and  MAF  Interaction 

The  PAA  also  interacts  with  MAF:  decisions  made  within  the  PAA  structure  are  informed  by 
strategic  goals  set  by  organizational  leaders  based  (in  part)  on  the  departmental  assessment  against 
the  Management  Accountability  Framework.  Annually,  the  organization’s  performance  is  assessed 
utilizing  the  MAF;  results  of  this  assessment  help  determine  the  organization’s  progress  in 
achieveing  short  term  and  long  term  strategic  goals;  the  organization’s  strategic  goals,  and  level  of 
progress  towards  them,  in  turn,  determine  the  programs  pursued  by  the  organization.  These 
programs  are  itemized  through  the  PAA  and  results  of  program  outcomes  are  utilized  as  reporting 
instruments  for  assessing  organizational  performance  in  terms  of  the  MAF.  Figure  5,  above,  depicts 
the  cyclcical  relationship  between  PAA  and  MAF. 

In  other  words,  the  MAF  articulates  the  government’s  management  performance  expectations 
overall  {how  we  manage),  while  the  PAA  acts  as  the  tool  for  identifying,  collecting  and  utilizing 
data  to  report  on  program  commitments  specifically  {what  we  manage). 

A  guidance  document  published  by  the  TBS  Information  Management  Strategies  Branch  offers  the 
following  explanation  of  the  relationship  between  MAF  and  MRRS: 

The  MRRS  is  an  inventory  of  and  information  on  specific  departmental 
programs.  It  includes  financial,  results  and  other  information  necessary  to 
analyze  specific  programs  and  make  decisions  on  these  programs  (e.g. 
where  the  Management  Accountability  Framework  questions  whether  an 
institution  has  an  effective  evaluation  function,  MRRS  asks  for  the  most 
recent  evaluation  report  on  a  program).  The  MRRS  uses  MAF  to  assist  in 
implementing  certain  elements,  and  the  intention  in  the  future  is  that 


Lee  McCormack,  Getting  the  Foundations  Right. 
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MRRS  program  information  should  include  most  MAP  indicators  (e.g.  the 
employment  equity  status  of  a  program  activity).^’ 

Because  the  MAP  works  in  tandem  with  the  PPA  and  MRRS  policy,  there  should  be  very  little 
additional  administrative  reporting  workload  for  managers  in  DRDC  in  providing  an  annual  MAP 
assessment  of  the  organization’s  program  activity  performance.  As  use  of  the  MAP  becomes  fully 
integrated  in  the  organization,  PAA  reporting  should  include  information  on  MAP  indicators,  as 
indicated  in  the  guidance  document  quoted  above,  so  that  additional  MAP  information  can  be 
extracted  directly  from  PAA  reporting. 

2.3  Issues  with  the  TB  MAP 

While  the  MAP  offers  a  useful  overview  of  the  Treasury  Board’s  expectations  for  senior 
management,  it  cannot  capture  the  details  of  decision  making  environments  and  pressures  which  are 
unique  to  each  institution,  nor  does  it  offer  a  methodology  for  assessing  the  quality  of  programming. 
In  this  sense,  the  TB  MAP  is  neither  a  replacement  for  assessing  DM  performance  through  the 
Committee  of  Senior  Officials  (COSO),  nor  an  alternative  to  program  evaluations,  although  it  does 
offer  valuable  perspectives  from  which  to  view  each  of  these  areas. 

As  a  guide  to  good  management  practices,  the  elements  focus  on  organizational  capacity  and 
capability  within  a  department. Plowever,  it  does  not  address  a  department’s  relationship  with  its 
Minister  and  it  does  not  discuss  horizontal  collaboration.^®  These  limitations  result  from  the  scope  of 
the  MAP,  which  provides  for  a  dialogue  between  the  Treasury  Board  Secretariat  and  Deputy 
Ministers  on  the  expectations  of  departmental  leadership 

The  MAP  was  originally  intended  as  a  ‘code  of  excellence’  type  document,^*  demonstrating  sound 
management  practices  which  would  rely  heavily  on  the  commitment  of  senior  managers  (DMs  and 
ADMs)  to  be  effective;  however,  as  the  framework  has  evolved  and  its  use  adopted  by  all 
departments,  it  has  become  “an  instrument  for  management  oversight,”  relying  more  heavily  on 
the  reporting  capabilities  of  subordinates.  Theoretically,  the  benefit  of  this  alternative  approach  is 
that  by  having  better  defined  and  mandatory  reporting  measures,  there  is  increased  and  better 
comparability  between  departments;  however,  as  noted  above,  each  department  has  unique 
environments  and  mandates  which  make  direct  comparison  difficult. 

One  drawback  in  the  shift  from  ‘code  of  excellence’  to  an  oversight  tool  is  that  by  redefining  MAP 
as  a  reporting  structure,  it  is  effectively  alienated  from  being  used  as  a  strategic  tool,  and  places 
more  burden  on  an  already  reporting-heavy  system: 


Treasury  Board  of  Canada  Secretariat,  “Guidance  on  Governance,  Accountability  and  Performance  Measurement  for  the 
Management  of  Information  in  Small  Agencies,”  2007,  available  from: 

httD://www.informationmanagement.gc.ca/docs/guid-orient/concepts/conceDts03  e.asp#  Toe  143324500.  (accessed  1 3 
August  2009) 

28  Lee  McCormack,  “Using  Evaluation  in  the  Canadian  Expenditure  Management  System”,  OECD  Presentation, 
Washington,  DC,  June  1 1-12,  2007,  available  from:  http://www.oecd.Org/dataoecd/21/9/39793340.pdf. 

29  Ivan  Blake,  2005. 

30  Ivan  Blake,  2005. 

31  Evert  Lindquist,  2009. 

32  Treasury  Board  of  Canada  Secretariat,  “MAE  Implementation,”  available  from:  http://www.tbs-sct.gc.ca/maf- 
crg/implementation-implementation/implementation-implementation-eng.asp. 
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“...the  cost  of  moving  too  far  in  this  direction  is  that  many  deputy 
ministers  will  create  sufficient  capability  to  credibly  feed  reporting 
requirements,  but  effectively  ‘quarantine’  it  [the  MAP]  from  strategic 
dialogue  and  decision-making  in  departments  focused  on  dealing  with 
pressing  challenges. 

Another  issue  with  the  MAP  is  that  it  can  portray  a  higher  level  of  departmental  accountability  than 
truly  exists.  This  weakness  stems  from  the  TBS’  dependence  on  each  department  to  self-report  by 
providing  information  on  MAP  measures,  a  lack  of  external  reviews,  as  well  as  a  lack  of  TBS 
capacity  to  provide  support  to  departments  after  the  assessment  to  close  gaps  in  MAP  performance 
areas.  Departments  which  become  better  at  responding  to  the  TB  MAP  reporting  structure  can  be 
given  improved  MAP  assessments  without  actually  improving. 

The  validity  of  MAP  measures  and  the  assessment  scoring  system  are  also  problematic  issues.  There 
is  no  information  available  to  explain  the  TBS’  methodology  in  selecting  indicators  and  measures 
for  each  Element.  The  departmental  assessment  against  each  measure  is  largely  dependent  on 
qualitative  judgments,  and  is  more  concerned  with  the  existence  and  implementation  of  policies  and 
frameworks  than  with  the  impact  of  those  policies  and  frameworks.  The  implicit  assumption  is  that 
the  mere  existence  of  policies  and  procedures  within  a  department  will  lead  to  improved  results. 

Two  weaknesses  in  the  scoring  system  are  that  each  indicator  is  given  equal  weight  in  the 
assessment  of  an  element  and  each  element  is  weighted  equally  for  all  departments.^®  Weak 
performance  against  a  particular  indicator  may  not  justify  a  weakened  assessment  in  the  element 
overall,  and  furthermore,  departmental  mandates  may  dictate  that  a  particular  element  be  weighted 
more  heavily^^  (the  service  element  for  Service  Canada,  for  example).  Note  that  there  is  no  overall 
score  for  the  MAP,  only  individual  elemental  assessments;  the  reader  is  left  to  draw  his/her  own 
conclusions  as  to  the  overall  departmental  performance  against  the  MAP. 

While  the  TB  MAP  was  not  intended  to  evaluate  DM  leadership  per  se,  judgment  is  implied, 
particularly  because  successful  performance  of  the  department  in  each  area  relies  heavily  on  senior 
management  leadership.  Not  only  is  it  awkward  for  subordinates  to  be,  in  effect,  evaluating  the 
DM’s  leadership  performance  when  conducting  a  MAP  assessment,  the  MAP  measures  do  not  take 
into  account  the  conflicting  priorities,  strategic  choices  and  unique  pressures  faced  by  each  DM.^* 

Pinally,  the  costs  associated  with  MAP  are  thought  to  be  quite  high  in  relation  to  its  benefits;  one 
conservative  estimate  places  the  cost  to  TBS  at  $42  million  annually.  This  figure  includes 
increased  overhead  costs,  people  costs  and  opportunity  costs. 

In  conclusion,  the  TB  MAP  is  a  good  checklist  against  which  departments  can  assess  their 
performance,  and  the  indicators  and  measures  are  adequate  starting  points  for  generic  assessments. 
Plowever,  each  department  or  agency  must  be  sensitive  to  their  particular  circumstances  when  self- 


Evert  Lindquist,  2009. 

Evert  Lindquist,  2009. 

Melvin  J.  Dubnick  and  Jonathan  B.  Justice,  Accounting  for  Accountability,  2004  Annual  Meeting  of  the  American 
Political  Science  Association,  discussion  paper,  September  2004,  p.  3. 

Evert  Lindquist,  2009. 

”  Evert  Lindquist,  2009. 

Evert  Lindquist,  2009. 

Evert  Lindquist,  2009. 
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assessing  against  the  MAP.  It  is  preferable  for  each  organization  to  institute  their  own  MAP,  based 
on  and  aligned  with  the  TB  MAP,  as  a  mechanism  for  assessing  performance  in  order  to  report 
effectively  to  superior  bodies  and  to  improve  their  own  management  performance. 

Table  3  outlines  the  six  main  problem  areas  with  the  TB  MAP  and  potential  strategies  to  mitigate 
these  problems  with  the  DRDC  MAP. 


Table  3:  MAF  Problem  Areas  and  Possible  Mitigation  Strategies 


MAF  Problem  Area 

Mitigation  Strategy  for  DRDC 

Loss  of  Strategic  Tool 

-Discuss  MAF  as  it  relates  to  organizational  priorities  at  RDEC  retreats 
-Select  relevant  information  sources  and  tools 

-Executive  summary  of  MAF  assessments  focuses  on  key  strategic  themes 
and  recommendations 

Reporting  Burden 

-Integrate  MAF  reporting  with  PAA  and  other  corporate  functions 

Appearance  of 
Accountability 

-Select  relevant  measures  and  information  sources 
-Document  methodology  so  that  the  same  approach  is  utilized  each  year 
-Set  goals  for  the  next  assessment  at  the  end  of  the  previous  assessment 
-Use  managers’  survey  to  incorporate  information  on  actual  working 
conditions 

Measures  and  Assessment 

-  Utilize  solid  academic  theory  as  basis  for  selection  of  measures  and 

development  of  survey  questions 

-Select  relevant  information  sources  and  tools 

Assessing  leadership 

-Do  not  utilize  MAF  assessments  as  a  performance  measurement  tool 
-Ensure  recommendations  are  aimed  at  organizational  level,  not  individuals 

Costs 

-Utilize  data  mining  techniques  to  get  relevant  information  from  pre-existing 
sources 

-Refine  other  reporting  systems  to  capture  MAF  related  measures  when  other 
data  is  collected 

-Select  relevant  measures  and  information  sources  to  cut  down  on 
unnecessary  reporting 
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3  Principles  and  Structure  of  the  DRDC  MAP 


3.1  Principles  of  the  DRDC  MAP 

A  DRDC  MAP  must  satisfy  the  following  conditions  in  order  to  be  adopted  willingly  by  the 
organization: 

Support  ADM(S&T)’s  contributions  to  the  TB  MAP  process 

Support  dialogue  that  clarifies  roles  and  responsibilities  throughout  the  chain-of-command 

Support  assessment  and  improvement  of  business  processes 

Exploit  existing  corporate  information  sources  where  possible 

Minimize  workload  where  direct  data  collection  from  managers  is  required 

A  DRDC  MAP  should  also  reflect,  and  thereby  promote,  DRDC’s  Mission,  Vision  and  Values,  as 
well  as  the  Defence  S&T  Strategy. 

Mission'*” 

DRDC's  mission  is  to  ensure  that  the  Canadian  Porces  are  technologically  prepared  and 
operationally  relevant  by: 

Providing  expert  science  and  technology  advice  to  the  Canadian  Porces  and  the  Department  of 
National  Defence; 

Conducting  research,  development  and  analysis  to  contribute  to  new  and  improved  defence 
capabilities; 

Anticipating  and  advising  on  future  science  and  technology  trends,  threats  and  opportunities; 

Engaging  industrial,  academic  and  international  partners  in  the  generation  and  commercialization 
of  technology;  and 

Providing  science  and  technology  for  external  customers  to  enhance  defence  science  and 
technology  capacity. 

Vision'** 

DRDC’s  vision  is  to  be  known  worldwide  as  the  best  in  science  and  technology  for  defence  and 
security. 


Direct  quote  from  DRDC,  Annual  Report  for  the  year  ending  3 1  March  2008:  Shaping  Defence  and  Security 
Capabilities  Through  Science  and  Technology,  p.  3. 

Direct  quote  from  DRDC,  Annual  Report  for  the  year  ending  31  March  2008,  p.  3. 
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Values'*^ 

DRDC’s  values  guide  how  we  accomplish  our  mission  and  maintain  excellence  in  science: 

Commitment:  We  demonstrate  dedication  and  pride  in  working  towards  our  vision. 

Client  Focus:  We  bring  excellence  to  clients,  both  internal  and  external,  by  focusing  efforts 
on  discovering  and  meeting  their  needs. 

Creativity  and  Innovation:  We  generate  innovative  solutions,  approaches,  products  and 
services  that  improve  the  status  quo. 

Leadership:  We  actively  and  enthusiastically  seek  to  exert  influence  and  originate  action  to 
achieve  our  goals. 

Professionalism  and  Integrity:  We  focus  our  effort  on  achieving  quality  results  and  we 
behave  in  an  honest,  ethical  manner,  dealing  with  others  respectfully  and  fairly. 

Trust  and  Respect:  We  are  open,  honest  and  responsible  in  our  relationships  and  we 
recognize  and  value  the  contributions  of  others. 

Teamwork:  We  demonstrate  effective  interpersonal  skills,  and  work  cooperatively  and 
productively  within  and  across  DRDC  to  achieve  common  goals. 

The  Defence  R&D  Strategy  includes  four  interdependent  areas,  each  associated  with  specific 
deliverables."^^  These  deliverables  support  the  core  elements  of  the  DRDC  MAP.  See  Table  4, 
opposite. 

The  Defence  S&T  Strategy  also  describes  six  attributes  the  Organization  must  cultivate  in  order  to 
be  successful  in  achieving  maximum  impact  in  the  identified  Action  Areas. All  six  are  dependent 
on  the  existence  of  a  solid  Governance  Structure.  The  Indicators  and  Measures  of  a  DRDC  MAP 
should  assess  the  strengths  of  these  six  attributes,  either  directly  or  indirectly. 

Linkage  -  through  the  Enterprise,  departmental  and  Canadian  Porces  organizations  are 
effectively  positioned  and  synchronized  to  contribute  to  the  direction,  delivery  and 
exploitation  of  S&T  outputs. 

Innovation  -  the  Enterprise,  with  its  external  partners,  produces  innovative,  cost-effective 
solutions  or  novel  alternatives  to  the  highest  priority  defence  and  security  needs  that  are 
appropriate  to  the  Canadian  context. 

Leverage  -  the  Enterprise  attracts  top-tier  national  and  international  S&T  partners  so  as  to 
leverage  their  resources,  knowledge,  experience  and  technology. 

Agility  -  the  Enterprise  is  appropriately  agile  so  as  to  identify  and  respond  effectively  to  new 
threats  and  opportunities  that  derive  from  the  global  advancement  of  S&T  or  from  the  rapidly 
evolving  defence  and  security  environment. 

Balance  -  the  S&T  outputs  produced  by  the  Enterprise  are  suitably  balanced  from  multiple 
perspectives.  These  outputs  address  the  prioritized  needs  from  the  perspectives  of  core 


Direct  quote  from  DRDC,  Annual  Report  for  the  year  ending  31  March  2008,  p.  3. 

Department  of  National  Defence,  Defence  S&T  Strategy:  Science  and  Technology  for  a  Secure  Canada,  December  2006, 

pp.  20-22. 

DND,  Defence  S&T  Strategy,  pp.  19-20. 
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departmental  proeesses,  Canadian  Forces  and  environment-specific  capability  requirements. 
They  address  the  needs  across  multiple  time  horizons,  from  responding  to  the  challenges  of 
today's  operations  to  shaping  the  Canadian  Forces  of  the  future.  Finally,  they  sustain  core 
internal  S&T  capabilities  across  as  broad  a  spectrum  as  resources  allow  while  ensuring  the 
critical  mass  necessary  in  each  area  to  produce  world  caliber  results. 

Excellence  -  the  Enterprise  consistently  produces  results  of  the  highest  possible  quality  and 
credibility  built  on  niche  world-caliber  internal  S&T  capabilities  and  strong,  effective  external 
partnerships. 


Table  4:  S&T  Strategy  Areas  and  Deliverables 


Action  Area 

S&T  Deliverables 

Key  MAE  Core  Elements 

Establish  an  integrated 
governance  mechanism 

Defence  S&T  Enterprise  Charter 

Accountability;  Risk  Management 

Defence  S&T  communications  plan 

Service 

Process  model  for  the  S&T 

Enterprise  Annual  Business  Cycle 

Accountability 

Aligned  Research,  Technology  and 
Analysis  Program  management 

Accountability 

Develop  a  "full-service" 
defence  S&T  capability 

S&T  capability  assessment  of  the 
S&T  Enterprise 

Policy  and  Programs 

Human  resources  plan  for  the  S&T 
Enterprise 

People 

Infrastructure  plan  for  the  S&T 
Enterprise 

Stewardship 

Build  strategic 
partnerships 

Process  model  for  managing 
partnerships 

Service 

Partnering  arrangements  among 

S&T  Enterprise  members 

People;  Service 

International  partnering  framework 
and  supporting  agreements 

Policy  and  Programs;  People 

Partnering  framework  for  industrial, 
academic  and  other  government 
departments 

Policy  and  Programs 

Establish  Enablers 

Information 

management/information 
technology  capability  to  support  the 
S&T  Enterprise 

Risk  Management;  Stewardship 

Harmonized  procurement  practices 
across  research,  development  and 
acquisition 

Stewardship 

Intellectual  property  management 
principles 

Stewardship 

Departmental  approach  to 
technology  insertion 

Risk  Management;  Stewardship 

DRDC  Atlantic  CR  2009-135 
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3.2  Structure  of  the  DRDC  MAF 


This  section  begins  with  the  presentation  of  the  DRDC  MAF  followed  by  a  vision  of  a  steady-state 
DRDC  MAF,  to  provide  a  strategic  overview  of  the  characteristics  of  a  DRDC  MAF  of  interest  to 
DRDC’s  executive  cadre:  the  purpose  of  the  MAF;  the  business  processes  of  the  MAF;  the 
participants  in  MAF  processes;  and  the  resources  required  by  the  MAF  processes.  An  overview  of 
the  Expectations  Tier  of  the  DRDC  MAF  is  then  presented,  followed  by  discussion  of  the  pillar  and 
core  elements  of  the  DRDC  MAF.  This  discussion  of  elements  includes  a  description  of  elements 
for  the  Indicators  and  Measures  Tiers  and  of  proposed  instruments  to  allow  data  collection  for  the 
measures. 

The  DRDC  MAF 

The  complete  DRDC  MAF  is  presented  here  in  TB  MAF  style.  Figure  6,  below,  depicts  the  DRDC 
MAF  Expectations  Tier;  Figure  7,  opposite,  depicts  the  DRDC  MAF  Indicators  Tier;  and  Figure  8, 
opposite,  depicts  the  DRDC  MAF  Measures  Tier. 


Governance 

and 

Strategic 

Direction 

The  OROC 
Charier,  the 
Defence 
Science  and 
Techno  logy 
Strategy  and 
theOND 
Business 
Planning 
process  ensure 
effective 
support  and 
functional  S&T 
authority  for 
Government, 
the  Canadian 
Forces,  and 
defence  and 
security 
partners 


Defence  R&D  Canada  Agency  Values 

Through  its  actions.  DRDC  Leadership  continually  reinforces  the  importance  of  the  agency’s 
values  and  of  defence  ethics^m  the  delivery  of  results  to  Canadian  forces  and  defence  and 

security  partners 


Policy  anil  Programs 

DRDC  research  and 
analytic  capacityis 
developed  and  sustained 
to  assure  high  quality 
policy  options,  program 
designand  advicetoONO/ 
CF.  Public  SafetyCanada. 
and  security  partners. 


People 

DRDC  has  the  people, 
vvork  environment, 
leadership  and  capadlyio 
providethe  best  in  science 
and  technologyfordefence 
and  security,  now  and  for 
the  future 


Service 

S  activities  are  client 
and  partner-centred, 
policiesand  programsare 
developed  to  address 
client  and  partner  needs, 
and  partnershipsare 
encouraged  and  effectively 
managed 


Risk  Moiioyemerit 

DRDC  prepares  for, 
manages  and  exploits 
risk  in  the  science  and 
techno  logyenvironmert. 


Stewoidship 

The  DRDC  Control 
regime  (assets,  money, 
people,  services, etc.)  is 
integrated  and  effecti/e. 

and  its  underlying 
principles  are  clear  to  al 
staff. 


Accountability 

Accountabilties  for  results 
are  clearly  assigned  and 
are  consistent  with 
resources,  and  delegations 
are  consistent  with 
capabilities. 


Learning,  Innovation  and  Change  Management 

DRDC  fosters  creativity,  innovalion,  and  change,  pronnoles  organizational  learning ,  values 
corporate  knowledge,  and  learns  from  its  performance 


Result! 

and 

Performance 


Relevant 
information  on 
results 
(internal, 
service  and 
program)  is 
gathered  and 
used  to  make 
decisions  in 
DRDC.  and 
agency  and 
public  reporting 
is  balanced , 
transparent, 
and  easy  to 
understand. 


Figure  6:  DRDC  MAF,  Expectations  Tier 
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Governance 

and 

Strategic 

Direction 


’Supportto  DM 
DNDIhrough  AOM 
(S&7) 

•M  anagement 
rramewor1<allgned 
to  strategic 
outcomes 

•The  right 
executive  team 

•Resutis-focused 
corporate  priorities 

•Strategic 

resource 

allocatlon/realioci^t 
ion  based  on 
performance 

•Integrated 

management 

agenda 

•Honzontai 
collaboration  I 

•Environmental  ' 
scanning 


Defence  RftD  Canada  Agency  Values 

•Exemplar/  conduct 

•Agency  values  are  reflected  by  the  re  alitiesand  culture  of  its  Centres 
•Value  s-based  managementpractices 


Policy  .iikI  Protjiains 

People 

Service 

•Worteforoe  is  suffkieflt  in  numbers 

Monitond ,  cuncinuously  irrifirD-ued 

-SuftliMd  IhMytiO  Cf plCdy  Ihd 

and  Bbdiief  whil*  reteeringih* 

culture  of  oonsuKitiofv,  rewpew  and 

Canadian  p^ulation 

servlec  quality 

challenge 

•WNtorkplMc  is  a  supportive  and 

•Leading  advice  on  teehnofcoy 

-Resdis  focused  poBoy  and  program 

mpectful  enwonment,  irtciuding 

*vploitil»n 

agendas  aligned  umh  government's 
twrijoiXal  priortlief 

respect  for  ofBcial  language 
requirements 

-Proactn*  engagemcfit  of  cBerrts 
ary]  partners 

•Effective  relaiionshpcis  w  Itft  clients  & 

•Sooi^  Evalu^kin  of  Impacts 

•Al  levels  of  management 
demonstrate  efketive  leadership 

•CoriSdenoeofth*  PDM 

•&nploy««s  a*  engaged  m  theirvork 

partners 

«lcnts  and  partners 

and  have  opportunlnes  to  team  arwi 
lyoui 

RlskMah<vjeinent 

•aifkif  minaetd  tvMhtniN*  «ent*Kt 
ONO  C^ont*  RIsIc  Prv^ 

•fiif k  Mana9ffn«nt  if  irrt*gr3(«cl  vilh 
ftlannin^  and  opcmiofts  sifXKtuws 
ihrwgtKKjt  tav*!;  ofth)*  Ag*ncy 
•Capacity  tusls  to  nppoft  a  Risk 
Management  Cuhur* 


Stewoi  (Iship 

Management  s^enw  prowle 
rylevani  infomtation  and  efrectivy 
teedback  onresourcys,  resvks  artd 
corrtrots 

•Ctfectrwy.  efic>*nt  ml  economicai 
pmject  manaeemerA 
•EnibtshediMemit  audit  iuncTiMi 
•Compidnoewlin  polieles, 
fy{iylat*«ns,  and  legiftation 


Accotintahility 

•Clear  accouniabllRlef  and 
mpons^HiitiesfcrdiMprocess  and 
results 

•Oelcgaltons  consistent  witk 

capaMiHs 

•$ubor4rvates'  goals  reflect  these 
assionedtothte  supervisors 


Learning,  Innovation  and  Change  Management 


■ORDC  iSB  learning  organizzrtioinwth  sirategievision 

•Organizational  ctituraanticipateschange.vaiuesinnovation.and  supports  Iran  storm  abon 
■DRDC  examines  its  owt  par  lormance  to  Id  entity  need  lor  change 
•Managers are  empovsered  through  delegation 

•Organizational  knoWedgaand  memory  captured  and  managed  as  strategic  resources 


Results 

and 

Performance 


-Monitoring  and 
reporting  of 
program,  seivice 
and  internal 
results 

•Integrated 
financial  and  non- 
financlal 
performance 
information  used 
in  decisionmaking 

•Reporting  is 
based  on 
measurable 
outcomes 

•Benchmark 
against  the  best 

•Transparent, 
timely  and 
accessible 
communications 
with  clients, 
parsers  and 
stakeholders 


Figure  7:  DRDC  MAF,  Indicators  Tier 


Governance 

and 

Strategic 

Direction 


•Confidence  of 
ADM  (S&T) 

•Coherent  policy 
agenda 

•Strong 

management 

cadre 

•Management 
framevrork 
supports  priority 
setting, 

reallocation, and 
alignmentwtth 
government- wide 
priorities 

•Change  agenda 
addressesall 
functional  areas 

•Organizational 
engagement  in 
administrative 
and  science- 
based  horizontal 
initiatives 


Defence  RAD  Canada  Agency  Values 

' Staff  I  d entify  with  DRDC  Values 


Ethical  decision  making  supported  by  the  Defence  Ethics  Program 


Organizational  performance  isevaiuaied  based  on  the  Defence  Ethics  Program 


Policy  anti  Pioyioins 

•OuMyof  PoiioMS  and  Programs 
•CornfwyhtrifnMI  Hyrnsn  R*F«4jr«« 
Mari*ii«m«rvi  fdan  fortM  pokey 
oommunitY 


•CorrvnhtnHwa  HRSiryt*0te  Plan 
in  plac* 

•Progra-ss  agHut  HRlargiAs 
Includng quMy  ofivotk  lif«.qtfl<i>< 
iangu»e«f .  tm^yt*  «quKy 
‘Progress  in  fMasurtrtg  a  improving 
•mployt*  *ngag«mMa 
•QusVy  «r  ■•adtrship 
•QuMy  of  labour  rclaiions 


•Cllonra  partrw  satlstactlon  1* 
m«as4ir«d  annually  againfl 
standads 

•CpltatM>rvtic>n  is  kaytodolivoting 
DRDCftrWc^f 
‘ORDCmakcs  Saiirifonnattkini 
roadilyavailatilt  to  slakohoidon 


Risk  Moihigeineiit 

•evld«n«4  that  Risk  Manaawrwnt  If 
int«grai«d  uilih  d40is*on  making , 
planning  and  reporting 
•Risk Manag«tr>«nttQ^. training,  and 
support  ar«  in  placo  for  staff 
Managtmtni  domonsinitf 
kadership  in  Risk  Managoirwit 
•Risk  Managotnont  approaoh 
compknontc  OND  cofporato  lisk 
proflk 

•Engagement  of  oMtmal  nakoheldors 
in  asfossing  /eomrmatieating  rtsks 


Stewoi  dstiip 


•ExptrMMurtsara  uiltitin  budget 
•Resotaoes  anproomd  and  used 
approfNiatelytoaohiev*  goals 
•Msets  arc  inwentorkd  and 
strate^allynunaged 
■infonnatlonif  ucillied  and  protected 
approfMtatcly 

■Oellirttiblef  ate  timely  and  effective 


Accoiiiit<il»iiity 
Management  goals  arc  idcniiilable 
and  measureabk 
•Duties  and  tasks  are  ckar  to 
suborMoies 

•Managers  and  nafTitave  apgropnate 
training  and  MhOiHy  to  undertake 
t  he  i  r  d^g  at  iofu 
•Afoimal  revk«  process  tracks 
progress,  delegations  and 
pertonnanec 

•AehMvemcrCof  suboninates'  goals 
supports  management  goals  and  the 
organifation^  Wslon 


Laarhtng,  IrinbVflrtion  aiid  Oiange  Managamarit 

•Progressandinvestmentin  organizational  learning 
•Piogressin  knowledge  manegement  practices 

‘Stakeholder/staff  perceptions  of  organizatiorel  adaptabiily,  change  and  innovation 
•Linkages  between  organizatiortal  performance  and  organizational  development  in  itiabves 


Results 

and 

Performance 


-Quality  of 
eusmessPian 
andAnnual 
Report 

•Staff  and  client 
survey  results 

•Prograssin 
strengthening 
financial  and 
program  results, 
and  performance 
measurement 

•Corporate 
monitoring  and 
review  of 
performance 

•Risk-based 
evaluation  plans 
(reviewed 
regularly)  and 
follow-up 

•Performance 
against  external 
benchmarks 


Figure  8:  DRDC  MAF,  Measures  Tier 
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Vision  of  a  Steady-State  DRDC  MAF 


The  DRDC  Management  Accountability  Framework  (MAF)  is  a  governance  tool 
that  facilitates  dialogue  between  managers  and  executive  leadership,  helping  to 
clarify  managers  ’  understanding  of  their  roles  and  responsibilities  and  providing 
a  framework  for  assessment  and  improvement  of  business  processes. 

The  Process 

The  DRDC  MAF  process  will  complement  the  DRDC  business  cycle.  The  DRDC  MAF  contains  1 0 
elements  but  ordinarily,  RDEC  will  begin  a  MAF  cycle  by  determining  which  elements  are  of 
greatest  current  interest,  because  of  Agency  or  Departmental  priorities.  The  resultant  MAF  cycle 
will  focus  on  those  elements,  but  may  also  address  others  on  an  overview  basis. 

The  MAF  process  will  collect  information  both  from  corporate  data  sources  and  from  a  survey  of 
DRDC  managers.  Corporate  staff  will  analyze  and  report  on  corporate  data  and  survey  results  at 
both  the  Agency  and  Centre  level,  in  the  context  of  the  elements  of  the  DRDC  MAF.  Results  of  the 
analysis  will  be  used  to  identify  both  areas  of  organizational  strengths  to  commend  and  weaknesses 
to  address,  including  identifying  areas  for  business  process  improvement.  Results  would  also 
support  the  commitments  of  the  ADM  (S&T)  to  the  DM  DND  and  TB  MAF. 

The  Participants 

RDEC  will  establish  the  priorities  of  any  given  MAF  cycle. 

DRDPO  will  be  the  OPl  for  the  DRDC  MAF.  DGSTO  and  DGRDCS  will  be  responsible  for 
provision  of  corporate  data,  and  DGMPRA  will  be  responsible  for  the  development  and  execution  of 
managers’  surveys  to  DRDPO  requirements.  Managers’  surveys  for  the  DRDC  MAF  would  be 
addressed  to  DRDC  Eevel  2  and  Eevel  3  Managers,  and  those  Eevel  4  functional  authorities  who  are 
members  of  Centre  management  committees.  DRDPO  would  analyze  and  report  the  results  of  the 
DRDC  MAF  cycle  to  RDEC. 

The  Resources 

DRDPO  will  plan  any  given  DRDC  MAF  cycle,  analyze  supplied  corporate  data  and  the  results  of 
the  managers’  survey,  and  report  the  results  to  RDEC  (17  person-weeks"^^).  DRDPO  may  also  use 
the  results  to  support  TB  MAF  requirements  (2  person-weeks). 

DGRDCS  and  DGSTO  will  provide  corporate  data  to  DRDPO  requirements  (5  person-weeks). 

DGMRPA  will  develop  and  implement  a  managers’  survey  to  DRDPO  requirements  and  deliver 
survey  tabulations  to  DRDPO  (8  A  person-weeks). 


These  resource  estimates  are  drawn  from  Section  4.2  of  Judy  Baroni  and  Warren  Nethercote,  A  DRDC  Management 
Accountability  Framework:  Final  Report,  DRDC  Atlantic  CR  2009-136. 
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DRDC  Level  2  and  Level  3  managers,  and  those  Level  4  functional  authorities  who  are  members  of 
Centre  Management  Committees  will  be  required  to  complete  the  managers’  survey  (4  person- 
weeks,  based  on  1  hour  per  manager). 


Annual  DRDC  MAF  Cycle 

The  DRDC  MAF  cycle  begins  in  February  with  planning  for  the  upcoming  MAF  analysis  based  on 
priorities  set  by  RDEC  the  previous  December.  Designated  DRDC  staff  will  identify  the  data 
necessary  to  report  on  the  organizational  performance  against  these  priorities. 

From  March  to  June,  staff  will  gather  relevant  data  and  information  measures  for  each  of  the  DRDC 
MAF  elements,  including,  but  not  limited  to,  those  sources  identified  in  this  report.  Many 
information  measures  will  be  available  through  pre-existing  databases;  some  data  can  easily  be 
retrieved  through  report  generation,  other  data  will  need  to  be  manually  extracted  from  documents, 
while  some  data  may  not  readily  exist  and  will  need  to  be  gathered  through  survey  or  audit.  As  the 
MAF  becomes  fully  implemented,  additional  reporting  requirements  can  be  integrated  with  pre¬ 
existing  reporting  structures  to  capture  additional  information. 

As  a  part  of  the  annual  MAF  analysis,  DRDC  managers  will  complete  a  survey  instrument  that 
examines  practices  relevant  to  the  elements  of  the  DRDC  MAF  being  examined  that  particular  year. 
The  survey  will  be  developed  and  tested  during  the  month  of  April.  Care  should  be  taken  to  avoid 
including  survey  items  which  can  be  found  through  corporate  sources  (ex:  training  participation).  A 
shared  question  bank  in  the  survey  instrument  for  the  DRDC  MAF  will  help  control  the  size  of  a 
survey  instrument  for  the  full  MAF,  thus  encouraging  survey  responses  that  are  carefully 
considered. 

The  survey  will  be  active  for  a  three  week  period  during  the  month  of  May.  Survey  participation  will 
be  mandatory  for  Level  2  and  3  managers,  and  selected  Level  4  functional  authorities. 

All  data,  including  survey  results,  will  be  analysed  during  the  month  of  June  in  anticipation  of 
completing  a  first  draft  of  findings  and  organizational  assessment  by  the  end  of  July  for  review  by 
RDEC;  a  second  draft  based  on  RDEC  feedback  is  deliverable  to  RDEC  by  the  end  of  August.  The 
final  report  of  the  DRDC  MAF  assessment  should  be  available  to  the  ADM  (S&T)  by  October  L', 
for  use  as  input  into  the  DND  reporting  requirements  for  the  TB  MAF  which  is  due  to  TBS  by 
November  F‘.  After  the  report  is  submitted,  DRDC  staff  would  complete  a  post-  mortem  of  the 
process  to  identify  areas  of  strength  and  opportunities  for  improvement;  staff  would  also  be 
available  for  consultation  by  DND.  The  MAF  assessment  is  posted  for  public  review  on  the  DRDC 
website  by  the  end  of  November.  In  December,  RDEC  should  identify  target  areas  for  the  next 
year’s  MAF  assessment,  based  in  part  on  the  previous  year’s  final  MAF  report. 

The  CEO/DRDC  will  use  the  results  of  analyses  to  support  the  Deputy  Minister’s  accountabilities  to 
the  Treasury  Board  Secretariat.  Eevel  2  managers  (Directors  General  and  COS/ADM(S&T)  will  use 
the  results  of  analyses  to  support  organizational  development  and  process  improvement.  Level  3 
managers  and  Level  4  functional  authorities  will  use  the  results  of  analyses  to  support  self- 
improvement. 

Table  5,  below,  orients  the  DRDC  annual  MAF  cycle  within  the  broader  TB  MAF  cycle. 
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Table  5:  Annual  MAF  Cycle'*^ 


August 
(Start  of  TB 
Cycle) 

September 

October 

November 

December 

January 

TB 

DM  is  contacted 
to  discuss  plans 
for  upcoming 

MAF  review  and 
to  get  names  of 
departmental 
contacts  for 
relevant  areas  of 
reporting 

Data  is 
requested 
from 
identified 

contacts 

Web  portal  is 
open  to 
receive 
departmental 
information 

All  requested 
information  is 
due  in  to  TB  by 
November  1; 
Analysis  begins 

Analysis 

DRDC 

Write  report;  2"“^ 
draft  by  end  of 
August:  changes 
to  RDEC  Retreat 

Write  final 
report 

DRDC  MAF 
analysis  due 
to  DND  by 

Oct  1 

Post-mortem; 
DRDC  MAF 
report  is  made 
publically 
available  on 
website 

RDEC 
identifies 
target  areas 
for  next 
DRDC 
round 

Staff 

available  to 

provide 

more 

information 
to  DND 

February 
(Start  of  DRDC 
Cycle) 

March 

April 

May 

June 

July 

TB 

Draft  to 
department;  4 
weeks  review 
period 

Analysis 

Final  report 
to  DM 

Post-mortem 
and  plan  for 
next  round 

Post¬ 
mortem 
and  plan 
for  next 
round 

Post-mortem 
and  plan  for 
next  round 

DRDC 

Begin  planning 
for  next  round; 
staff  available  to 
provide  more 
information  to 

DND 

Gather 
relevant 
information 
and  data 

Gather 
relevant 
information 
and  data; 
develop  and 
test  survey 

Gather  relevant 
information  and 
data; 

management 
survey  is  active 
for  a  3  week 
period 

Gather 

relevant 

information 

and  data; 

survey  and 

data 

analysis 

Write  report; 

1  **  draft  to 
RDEC  for 
RDEC 

Retreat  by 
end  of  July 

Treasury  Board  of  Canada  Secretariat,  MAF  Assessment  -  Round  VI,  available  from:  http://www.tbs-sct.gc.ca/maf- 
crg/assessments-evaluations/2008/assessments-evaluations-eng.asp;  Citizenship  and  Immigration  Canada,  Integrated 
Planning:  the  View  from  CIC,  presentation,  PPX  breakfast  session,  December  2,  2008,  available  from: 
httti://www.ppx.ca/PPXevents/Archive/Dec2  08/Dec2IntegratedPlanningfAllisonFortinENG.Ddf:  Evert  Lindquist,  2009; 
and  Virginie  Ethier,  2007. 
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4  The  Pillar  Elements 


There  are  four  pillar  elements  of  the  DRDC  MAP:  Defence  R&D  Canada  Agency  Values; 
Governance  and  Strategic  Direction;  Results  and  Performance;  and  Learning,  Innovation  and 
Change  Management.  These  four  pillar  elements  warrant  different  treatment  from  the  six  core 
elements.  The  core  elements  apply  to  everyone  in  the  organization,  but  are  especially  relevant  to 
line  managers,  whose  activities  determine  the  success  of  DRDC  relative  to  MAP  measures.  In  effect, 
the  core  elements  set  the  standards  for  the  performance  of  management.  The  pillar  elements  are 
different,  in  that  they  set  the  strategic  environment  for  management. 

Defence  R&D  Canada  Agency  Values  set  the  very  character  of  DRDC,  defining  the  type  of 
environment  and  the  behaviour  and  attitudes  of  employees  that  best  represent  the  organization. 
Governance  and  Strategic  Direction  describes  the  strategic  policy  basis  for  DRDC  management 
activities,  ensuring  that  management  activities  respond  to  policy-based  needs.  Results  and 
Performance  identifies  the  need  for  evidence-based  decision-making  and  evaluation  of  program 
performance.  Pinally,  Learning,  Innovation  and  Change  Management  describes  the  environment  in 
which  managers  manage.  If  the  core  elements  control  how  managers  manage,  then  the  pillar 
elements  set  the  environment  in  which  they  manage.  The  provision  of  that  environment  is  the 
purview  of  DRDC’s  executive  leadership,  rather  than  of  managers.  Thus,  whereas  the  core  elements 
of  the  DRDC  MAP  focus  on  Level  3  and  4  managers,  the  pillar  elements  should  focus  on  Level  1 
and  2  managers,  whose  behaviour  and  attitudes  set  the  standard  for  Level  3  and  4  managers.  Of 
course.  Level  3  and  4  managers  cannot  ignore  the  pillar  elements,  but  the  pillar  elements  represent 
enablers  or  constraints,  rather  than  measures  of  performance. 

The  development  of  the  pillar  elements  relied  heavily  upon  the  results  of  application  of  TB  MAP  to 
the  Department  of  National  Defence,  together  with  corporate  documents.  This  approach  reflected 
the  strategic  nature  of  the  pillar  elements  and  the  need  to  establish  clear  linkages  with  the  superior 
TB  MAP. 

4.1  Defence  R&D  Canada  Agency  Values 

Defence  R&D  Canada’s  values,  together  with  defence  ethics  standards,  position  DRDC  to  respond 
to  changing  world  and  organizational  circumstances  where  management  by  prescriptive  rule -sets 
will  often  be  found  wanting.  They  represent  DND  and  DRDC’s  implementation  of  the  Values  and 
Ethics  Code  for  the  Public  Service."^’ 

The  Values  and  Ethics  Code  for  the  Public  Service  is  reflected  in  DRDC’s  values.  Table  6  shows 
the  correspondence  between  the  two. 


Treasury  Board  of  Canada  Secretariat,  Values  and  Ethics  Code  for  the  Public  Service,  available  from:  http://www.tbs- 
sct.gc.ca/pubs  pol/hrpubs/tb  851/vec-cve-eng.asp. 


DRDC  Atlantic  CR  2009-135 


23 


Table  6:  Values  and  Ethics  in  DRDC  and  the  Public  Service 


DRDC  Values 

Public  Service  Values 

Democratic  Values:  Helping 
Ministers,  under  law,  to  serve 
the  public  interest 

Professional  Values: 

Serving  with  Competence, 
excellence,  efficiency, 
objectivity  and  impartiality 

Ethical  Values:  Acting  at  all 
times  in  such  a  way  as  to 
uphold  the  public  interest 

People  values: 

Demonstrating  respect, 
fairness  and  courtesy  in  their 
dealings  with  both  citizens 
and  fellow  public  servants 

Commitment:  We  demonstrate  dedication 
and  pride  in  working  towards  our  vision 

X 

X 

X 

Client  Focus:  We  bring  excellence  to  clients, 
both  internal  and  external,  by  focusing  efforts 
on  discovering  and  meeting  their  needs 

X 

X 

X 

Creativity  and  Innovation:  We  generate 
innovative  solutions,  approaches,  products 
and  services  that  improve  the  status  quo. 

X 

Leadership:  We  actively  and  enthusiastically 
seek  to  exert  influence  and  originate  action  to 
achieve  our  goals 

X 

X 

X 

Professionalism  and  Integrity:  We  focus  our 
effort  on  achieving  quality  results  and  we 
behave  in  an  honest,  ethical  manner,  dealing 
with  others  respectfully  and  fairly. 

X 

X 

X 

X 

Trust  and  Respect:  We  are  open,  honest  and 
responsible  in  our  relationships  and  we 
recognize  and  value  the  contributions  of 
others 

X 

X 

Teamwork:  We  demonstrate  effective 
interpersonal  skills,  and  work  cooperatively 
and  productively  within  and  across  DRDC  to 
achieve  common  goals. 

X 
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DRDC  Agency  Values:  Expectations 

Through  its  actions,  DRDC  Leadership  continually  reinforces  the  importance  of 
the  agency’s  values  and  of  defence  ethics  in  the  delivery  of  results  to  the 
Canadian  Forces  and  defence  and  security  partners. 

The  corresponding  statement  in  the  TB  MAP  is: 

Through  their  actions,  departmental  leaders  continually  reinforce  the  importance 
of  public  service  values  and  ethics  in  the  delivery  of  results  to  Canadians  (e.g. 
democratic,  professional,  ethical  and  people  values). 

Figure  9,  below,  depicts  the  Expectations  statement  for  DRDC  Agency  Values  in  the 
DRDC  MAP. 


DRDC  Agency  Values 

Througli  its  actions,  DRDC  Leadership 
continually  leinfoices  the  inq)oitance  of  the 
agency’s  values  and  of  defence  ethics  in  the 
dehveiy  of  results  to  the  Canadian  Forces 
and  defence  and  security  partners. 


II 


Figure  9  :  DRDC  Agency  Values,  Expectations  Tier 


This  Expectations  statement  has  been  amended  from  the  Cycle  1  DRDC  AMP,  changing  “through 
the  delivery  of  results”  to  “in  the  delivery  of  results”  and  adding  reference  to  defence  ethics  to 
maintain  alignment  with  the  TB  MAP. 


DRDC  Agency  Values:  Indicators 

The  TB  MAP  employs  three  indicators  for  Public  Service  Values: 

Exemplary  conduct:  This  indicator  is  clear  and  applicable  to  the  DRDC  context.  It  has  been 
transferred  directly  to  the  indicator  Exemplary  conduct. 

Public  service  values  tailored  to  realities/culture  of  department:  This  indicator  has  been  adjusted  to 
reflect  the  DRDC  environment  as  Agency  Values  are  reflected  in  the  realities  and  cultures  of  its 
Centres. 

Values-based  management  practices:  This  indicator  is  clear  and  applicable  to  the  DRDC  context.  It 
has  been  transferred  directly  to  the  indicator  Values-based  management  practices. 
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Figure  10,  below,  depicts  the  Indicators  for  DRDC  Agency  Values  in  the  DRDC  MAF. 


DRDC  Agency  Values 

Exen^laiy  conduct 

Agency  values  are  reflected  by  the  realities 
and  cultures  of  its  Centres. 

Values-based  management  practices 

Figure  10  :  DRDC  Agency  Values,  Indicators  Tier 


II 


DRDC  Agency  Values:  Measures 

Customized  public  service  values  statement  and  ethical  guidelines  regularly  discussed  with  all  staff: 

this  measure  has  been  re-worded  as  Staff  identify  with  DRDC  Values.  DRDC  has  Agency  Values 
(shown  above  in  Table  5)  which  satisfy  the  customized  values  statement  requirement.  Whether 
DRDC’s  ethical  guidelines  are  ‘regularly  discussed  with  all  staff  is  imputed  through  the  level  of 
staff  identification  with  the  organizational  values. 

Sound  advisory  and  recourse  mechanism  in  place:  this  measure  has  been  adjusted,  to  better  reflect 
DRDC  culture,  as  Ethical  decision  making  supported  by  the  Defence  Ethics  Program.  The 
Department  of  National  Defence  has  a  well-established  Defence  Ethics  Program"'*,  to  which  DRDC 
subscribes,  as  a  DND  Level  1  organization.  This  program  is  the  mechanism  for  ethics  and  values 
advisory  and  recourse  in  the  organization.  DSTER  is  DRDC’s  Level  1  Ethics  Coordinator. 

Orientation,  learning  and  other  tools  to  support  staff:  this  measure  has  been  folded  into  the  previous 
measure  Ethical  decision  making  supported  by  the  Defence  Ethics  Program.  Defence  Ethics 
training  is  mandatory  for  all  new  staff,  DND  and  CF-wide.  DAODs  and  information  materials  are 
available  for  staff  support,  in  addition  to  Level  1  coordinators,  or  local  level  1  representatives. 

Staff  assessment  of  organizational  performance  against  PS  values  and  ethics:  this  statement  is 
captured  utilizing  DRDC  vernacular  as  Organizational  performance  is  evaluated  based  on  the 
Defence  Ethics  Program. 


Figure  11,  opposite,  depicts  the  Measures  for  DRDC  Agency  Values  in  the  DRDC 
MAF. 


See  Department  of  National  Defence,  Defence  Administration  Orders  and  Directives,  DAOD  7023-0, 
Defence  Ethics,  2003,  available  from:  http://www.admflncs.forces.gc.ca/dao-doa/7000/7023-0-eng.asp. 
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DRDC  Agency  Values 

Staff  identify  with  DRDC  Values 

Ethical  decision  making  supported  by  the 
Defence  Ethics  Program 

Organizational  performance  is  evaluated 
based  on  the  Defence  Ethics  Program 


II 


Figure  11  :  DRDC  Agency  Values,  Measures  Tier 


For  the  most  part,  measures  can  be  assessed  using  existing  instruments.  Commitment  to  the  Defence 
Ethics  program  can  be  readily  inferred  by  examining  PeopleSoft®-based  training  records:  for 
example,  have  all  new  staff  received  defence  ethics  training  within  an  acceptable  time  of  their  being 
taken  on  strength  (TOS)?  Similarly,  examining  the  activities  of  the  DRDC  Ethics  Coordinator"*^ 
would  offer  insight  into  the  ethical  health  of  DRDC.  The  2008  Public  Service  Employee  Survey 
includes  ethics-related  questions  and  with  over  500  respondents  for  DRDC,^°  and  this  should  offer  a 
reasonable  representation  of  DRDC’ s  ethical  health.  A  potentially  stronger  instrument  is  the 
Defence  Ethics  Survey,  delivered  every  three  years,  although  past  versions  have  only  been 
addressed  to  DND  employees  and  CF  Members  with  addresses  on  the  DWAN/DIN.  Defence  R&D 
Canada  employees  were  not  explicitly  identified  in  previous  Defence  Ethics  Surveys;  instead,  they 
were  grouped  with  ‘other  commands  and  Eevel  Is’.  Going  forward,  administration  to  DRDC  via 
DRENet  would  be  feasible,  but  would  likely  require  reprogramming  on  a  DRENet  or  web-based 
utility.  On  an  ongoing  basis,  ethics-related  instruments  might  be  expanded  to  include  workshops  or 
focus  groups,  although  these  two  are  perhaps  more  diagnostic  or  even  corrective  measures  than 
assessment  tools.  At  the  manager  level,  the  Managers’  Workshop  would  provide  a  good  opportunity 
for  annual  ethical  ‘exercise’. 


Table  7  presents  examples  of  information  which  can  be  used  as  evidence  of  each  measure. 


Currently  the  Director,  R&D  Business  Affairs. 

Treasury  Board  of  Canada  Secretariat,  2008  Public  Service  Employee  Survey  Results:  Organizational  Report  for 
Department  of  National  Defence  Assistant  Deputy  Minister  Science  and  Technology  Group,  available  from: 
http://www.tbs-sct.gc.ca/pses-saff/2008/results-resultats/res- 

eng.aspx?cd=&o  1  =03&o2=0 1 9&o3=000&o4=000&o5=000#tphp. 

Most  recently  in  2007;  see  Kyle  Fraser,  The  2007  Defence  Ethics  Survey:  Summary  of  the  Overall  CF  and  DND 
Findings  for  Decision  Makers,  DRDC  CORA  TN  2008-016,  August  2008. 
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Table  7:  Evidence  and  Instruments  for  Defence  R&D  Canada  Agency  Values 


Measure:  Staff  identify  with  DRDC  Values 

Evidence 

Source/  Instruments 

•  Public  Service  Values  and  Ethics  Infrastructure  in  Place, 
Understood  and  Effective 

•  Organizational  Culture  is  reflective  of  DRDC  Values  and 
Ethics 

•  Extent  to  which  leaders  foster  a  culture  of  respect  and 
integrity 

•  Feedback  from  employees  on  fairness,  respect, 
satisfaction  and  engagement; 

•  Values  and  ethics  exercise 
at  DRDC  Managers’ 
Workshop 

•  Focus  groups  or  surveys  of 
staff 

•  Focus  groups  or  surveys  of 
clients 

Measure:  Ethical  decision  making  supported  by  the  Defence  Ethics  Program 

Evidence 

Source/  Instruments 

•  Senior  management  demonstrates  strong  Values  and 
Ethics 

•  Organizational  performance  aligns  with  Values 

•  Eeadership  communication  with  employees  about 
expected  ethical  behaviour  and  public  service  values; 

•  Selection,  evaluation,  promotion  and  discharge  of 
leaders  based  on  their  conduct  with  respect  to  values 
and  ethics. 

•  Appropriate,  accessible  avenues  for:  employee  advice, 
reports  of  wrongdoing  and  resolution  of  conflict 

•  Defence  Ethics  Training 
statistics  from  PeopleSoft® 

•  DRDC  Ethics  Coordinator 
activity  statistics 

Measure:  Organizational  performance  is  evaluated  based  on  Defence  Ethics  Program 

Evidence 

Source/  Instruments 

•  Benchmark  results  and  implemented  improvements 

•  Extent  to  which  the  organization  maintains  and  promotes 
appropriate,  accessible  avenues  for  disclosure  of 
wrongdoing 

•  Trends  in  management  and  program  irregularities, 
regularly  reported  on  and  reviewed  by  management. 

•  Review  of  Ethics 

Coordinator  activities 

•  Focus  groups  or  surveys  of 
staff 
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4.2  Governance  and  Strategic  Direction 


Governance  and  Strategic  Direction:  Expectations 

The  DRDC  Charter,  the  Defence  Science  and  Technology  Strategy,  and  the  DND 
business  planning  process  ensure  effective  support  and  functional  S&T  authority 
for  Government,  the  Canadian  Forces  and  defence  and  security  partners. 

The  corresponding  statement  in  the  TB  MAP  is: 

The  essential  conditions  -  internal  coherence,  corporate  discipline  and 
alignment  to  outcomes  -  are  in  place  for  providing  effective  strategic  direction, 
support  to  the  minister  and  Parliament,  and  the  delivery  of  results 

The  DRDC  statement  identifies  key  elements  -  the  ‘essential  conditions’  -  that  serve  to  ensure 
effective  support  and  position  ADM(S&T)  as  the  Department’s  functional  authority  for  defence 
S&T.  The  DRDC  Charter  establishes  the  raison  d’etre  of  the  organization,  its  relationship  to  DND, 
the  CF  and  stakeholders,  and  governance,  organization  and  accountability.^^  The  Defence  S&T 
Strategy  provides  the  framework  for  delivery  of  S&T  in  the  defence  and  security  environment,  both 
by  DRDC  and  by  its  partners.  Finally,  DRDC’s  integration  in  the  DND  business  planning  process 
ensures  that  DRDC’s  activities  will  support  the  Strategic  Outcomes  of  DND’s  Program  Activity 
Architecture. 

Figure  12,  below,  depicts  the  Expectations  statement  for  Governance  and  Strategic 
Direction  in  the  DRDC  MAF. 


Governance  and  Stra^ic  Direction  a 


The  DRDC  Charter,  the  I>efence  Science  and 
Technology  Strategy,  and  the  DND  business 
planningpiocess  ensure  effective  support  and 
functional  S&T  authority  for  Government,  the 
Canadian  Forces  and  defence  and  security 
partners. 


1 

1 

Figure  12:  Governance  and  Strategic  Direction,  Expectations  Tier 


Governance  and  Strategic  Direction:  Indicators 

The  TB  MAF  employs  eight  indicators  for  Governance  and  Strategic  Direction: 


Defence  Research  and  Development  Canada,  Framework  Document,  approved  by  Treasury  Board  Ministers  July  27, 
2000. 

Department  of  National  Defence,  Defence  S&T  Strategy:  Science  and  Technology  for  a  Secure  Canada. 
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Support  to  Minister,  Cabinet  and  Parliament:  This  indicator  has  been  changed  to  Support  to  DM 
DND  through  ADM  (S&T)  to  better  reflect  DRDC’s  reporting  structure. 


Management  iramework  aligned  to  strategic  outcomes:  This  indicator  can  be  directly  applied  to  the 
DRDC  MAP  as  Management  framework  aligned  to  strategic  outcomes. 

The  right  executive  team:  This  indicator  can  be  directly  applied  to  the  DRDC  MAP  as  The  right 
executive  team. 

Results-focused  corporate  priorities  :This  indicator  can  be  directly  applied  to  the  DRDC  MAP  as 
Results-focused  corporate  priorities. 

Strategic  resource  allocation/reallocation  based  on  performance:  This  indicator  can  be  directly 
applied  to  the  DRDC  MAP  as  Strategic  resource  allocation/reallocation  based  on  performance. 

Integrated  agenda  for  management  excellence:  This  indicator  has  been  reworded  for  clarity  as 
Integrated  management  agenda. 

Plorizontal  collaboration  :This  indicator  can  be  directly  applied  to  the  DRDC  MAP  as  Horizontal 
collaboration. 

Environmental  scanning:  This  indicator  can  be  directly  applied  to  the  DRDC  MAP  as 
Environmental  scanning. 

Pigure  13,  below,  depicts  the  Indicators  for  Governance  and  Strategic  Direction  in  the  DRDC  MAP. 


Govemanfre  and  Stra^ic  Direction 

Support  to  DM  DND  throng  ADM  (S&T) 
Management  frameworkal^ed  to  strategic  outcomes 
Theriot  executive  team 
Results-fijcused  coipnate  piicuities 

Stral^c  resource  allocation/reallocatioubased  ou 
performance 

Int^rated  mana^ment  ^^da 
Horizontal  collaboraticn 
Environmoital  sranning 


Figure  13:  Governance  and  Strategic  Direction,  Indicators  Tier 


Governance  and  Strategic  Direction:  Measures 

The  TB  MAP  employs  six  measures  for  Governance  and  Strategic  Directions,  shown  below,  along 
with  their  DRDC  MAP  counterparts: 


30 


DRDC  Atlantic  CR  2009-135 


Minister’s  confidence  in  departmental  support:  This  measure  has  been  changed  to  Confidence  of 
ADM  (S&T)  to  better  reflect  DRDC’s  reporting  structure. 

Perceived  coherence  of  policy  agenda:  This  measure  can  be  applied  to  the  DRDC  MAP,  with  a 
slight  rewording,  as  Coherent  policy  agenda;,  the  assessment  of  coherence  is  intrinsically 
subjective,  thus  ‘perceived  coherence’  is  redundant. 

Strength  of  the  management  team:  This  measure  has  been  changed  to  Strong  management  cadre  to 
better  reflect  DRDC’s  vernacular. 

Corporate  management  framework  used  for  priority  setting,  reallocation,  and  alignment  to 

government-wide  priorities:  this  measure  has  been  re-worded  for  clarity  as  Management  framework 
supports  priority  setting,  reallocation,  and  alignment  with  government-wide  priorities 

Management  improvement  agenda  integrating  HR,  comptrollership,  service,  etc.:  this  measure  has 
been  captured  more  succinctly  in  the  statement  Change  agenda  addresses  all  functional  areas. 

Leadership/participation  in  PS-wide  initiatives:  this  measure  has  been  changed  to  Organizational 
engagement  in  administrative  and  science-based  horizontal  initiatives  to  better  reflect  DRDC’s  S&T 
orientation  within  Government. 

Figure  14,  below,  depicts  the  Measures  for  Governance  and  Strategic  Direction  in  the  DRDC  MAP. 


Governance  and  Strat^c  Dtrection 

ConfidoiceofADM  (S&T) 

Cohorait  pcdicy  a^»da 
Strong  man^^oit  cadre 

Management  fiameworksnpports  pdodty  settiig,  reallocadcn, 
and  aligjnmeat  with  govonmait-wide  priodties 

Change  %enda  addresses  all  liinctional  areas 

Organizational  eng^ement  in  administrative  and  scimce- 
based  horizontal  initiatives 


Figure  14:  Governance  and  Strategic  Direction,  Measures  Tier 
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Table  8  presents  examples  of  information  which  can  be  used  as  evidence  of  each  measure. 


Table  8:  Evidence  and  Instruments  for  Governance  and  Strategic  Direction 


Measure:  Confidence  of  ADM  (S&T) 

Evidence 

Source/  Instruments 

•  ADM  (S&T)  confidence 

•  Reports  of  ADM  (S&T) 

•  DRDC  Level  2  PMA 
statistics 

Measure:  Coherent  policy  agenda 

Evidence 

Source/  Instruments 

•  DRDC  Business  Plan  to  show  coherence  with  PAA 

•  DRDC  PGAs  to  show  coherence  with  PAA  sub  and 
sub-sub-activities 

•  Presence  of  a  risk-based  corporate  plan 

•  DRDC  Business  Plan  to 
show  links  to  PAA 

•  DRDC  PGAs  to  show 
coherence  with  sub-  and  sub- 
sub-activities  in  PAA 

•  DRDC  Corporate  Risk 

Profile 

Measure:  Strong  management  cadre 

Evidence 

Source/  Instruments 

•  Quality  of  Performance  Management  Agreements 

•  Performance  Management  Agreements  support 
organizational  decision  making 

•  Analysis  of  DRDC  PMAs 

•  DRDC  MAE  survey  for 
management  cadre 
performance  against  core 
elements 

32 


DRDC  Atlantic  CR  2009-135 


Measure:  Management  framework  supports  priority  setting,  reallocation,  and  alignment  with 
government-wide  priorities 

Evidence 

Source/  Instruments 

•  Extent  to  which  the  organization's  Strategic  Outcomes 
and  Program  Activity  Architecture  are  consistent  with 
its  mandate 

•  Clarity  and  measurability  of  the  organization's  strategic 
outcomes 

•  Eevel  of  completeness  of  the  Program  Activity 
Architecture  and  its  alignment  to  the  organization's 
strategic  outcomes 

•  Accountabilities  of  senior  officials  and  internal  decision¬ 
making  structures  are  aligned  to  the  organization's 
strategic  outcomes 

•  Corporate  PMA  analysis 

•  DRDC  MAE  survey  for 
management  cadre 
performance  against  core 
elements 

•  Business-line  planning 
allows  alignment  of  PGA 
resources  to  strategic 
agenda 

•  Comparison  of  successive 
Business  Plans  for  evidence 
of  priority-setting  and 
reallocation 

•  Partner  Group  Agreements 
(PGA)  form  the  basis  for 
results-focused  S&T 

•  Do  PGAs  reflect  Business 
Plan  priorities? 

Measure:  Organizational  engagement  in  administrative  and  science-based  horizontal  initiatives 

Evidence 

Source/  Instruments 

•  Evidence  of  activities  such  as  ADMs  Committee  on 

S&T  and  other  horizontal  S&T  initiatives 

•  Eevel  of  organizational  engagement  (e.g. 
appropriateness,  leadership,  resource  commitment)  on 
horizontal  policy  and  program  initiatives 

•  Evidence  of  activities  (S&T 
ADMs,  CRTI,  etc.) 

•  DRDC’s  Technology  Watch 
program  provides  warning 
of  disruptive  technologies 
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4.3  Results  and  Performance 


Results  and  Performance:  Expectations 

Relevant  information  on  results  (internal,  service  and  program)  is  gathered  and 
used  to  make  decisions  in  DRDC,  and  agency  and  public  reporting  is  balanced, 
transparent,  and  easy  to  understand. 

The  DRDC  expectations  statement  closely  reflects  that  of  the  TB  MAP,  shown  below: 

Relevant  information  on  results  (internal,  service  and  program)  is  gathered  and 
used  to  make  departmental  decisions,  and  public  reporting  is  balanced, 
transparent,  and  easy  to  understand. 


Figure  15,  below,  depicts  the  Expectations  Statement  for  Results  and  Performance  in 
the  DRDC  MAP. 


Results  and  Performance 


Relevant  infoimation  on  results  (internal, 
service  and  program)  is  gathered  and  used 
to  make  decisions  in  DRDC,  and  agency 
and  public  reporting  is  balanced, 
transparent,  and  easy  to  understand. 


Figure  15:  Results  and  Performance,  Expectations  Tier 


Results  and  Performance:  Indicators 

The  TB  MAP  employs  five  indicators  for  Governance  and  Strategic  Directions,  shown  below,  along 
with  their  DRDC  MAP  counterparts  : 

Corporate  monitoring  and  reporting  of  program,  service  and  internal  results:  This  indicator  can  be 
applied  to  the  DRDC  MAP  with  a  slight  rewording.  Monitoring  and  reporting  of  program,  service 
and  internal  results. 

Integrated  financial  and  non-financial  performance  information  used  in  corporate  decision  making: 

This  indicator  can  be  applied  to  the  DRDC  MAP  with  a  slight  rewording.  Integrated  financial  and 
non-financial  performance  information  used  in  decision-making. 

Departmental  reporting  based  on  measurable  outcomes:  This  indicator  can  be  applied  to  the  DRDC 
MAP  with  a  slight  rewording.  Reporting  is  based  on  measurable  outcomes. 
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Benchmark  against  the  best:  This  indicator  can  be  directly  applied  to  the  DRDC  MAP,  keeping  in 
mind  the  Agency  benchmarks  both  its  corporate  functions  and  its  Science  and  Technology 
functions. 

Transparent,  timely  and  accessible  communications  with  citizens  and  Parliament:  DRDC  has  a 
narrower  target  audience  for  its  reports  and  publications.  Due  to  the  nature  of  the  Agency’s  work, 
some  reports  and  communications  may  not  be  readily  available  to  citizens.  This  indicator  is  captured 
in  the  statement  Transparent,  timely  and  accessible  communications  with  clients,  partners  and 
stakeholders 

Figure  16,  below,  depicts  the  Indicators  for  Results  and  Performance  in  the  DRDC  MAP. 


Results  and  Performance 


Momtoiiogandiepoiting  of  program,  service  and 
internal  results 


Integrated  financial  and  non-financial  perlbtiiiance 
infoimaticmused  in  decision  making 
Rqpoitingis  based  on  measurable  outcomes 
Benchmark  against  the  best 

Transparent,  timely  and  accessible  communications 
with  clients,  partners  and  stakeholders 


Figure  16:  Results  and  Performance,  Indicators  Tier 


Results  and  Performance:  Measures 

The  TB  MAP  employs  five  measures  for  Governance  and  Strategic  Directions,  shown  below,  along 
with  their  DRDC  MAP  counterparts  : 

Quality  of  Reports  on  Plans  and  Priorities  and  Departmental  Performance  Report:  this  measure  has 
been  adjusted  as  Quality  of  Business  Plan  and  Annual  Report  to  reflect  DRDC’s  status  as  an 
Agency. 

Staff  and  client  survey  results:  This  indicator  can  be  directly  applied  to  the  DRDC  MAP  as  Staff  and 
client  survey  results. 

Progress  in  strengthening  financial  and  program  results,  and  performance  measurement:  This 

indicator  can  be  directly  applied  to  the  DRDC  MAP  as  Progress  in  strengthening  financial  and 
program  results,  and  performance  measurement. 

Corporate  monitoring  and  review  of  performance:  This  indicator  can  be  directly  applied  to  the 
DRDC  MAP  as  Corporate  monitoring  and  review  of  performance. 
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Risk-based  evaluation  plans  (reviewed  regularly)  and  follow-up:  This  indicator  can  be  directly 
applied  to  the  DRDC  MAP  as  Risk-based  evaluation  plans  (reviewed  regularly)  and  follow-up. 

Performance  against  external  benchmarks:  This  indicator  can  be  applied  to  the  DRDC  MAP  as 
Performance  against  external  benchmarks. 

Pigure  17,  below,  depicts  the  Measures  for  Results  and  Performance  in  the  DRDC  MAP. 


Results  and  Performance 

Quality  ofBusmess  Plan  and  Annual  Report 
Staff  and  client  survey  results 

Progress  in  strengthening  financial  and  program 
results,  and  perfimnance  measurement 
Corporate  monitoringand  review  of  performance 


Risk-based  evaluation  plans  (reviewed  regularly) 
and  follow-up 

Performance  against  external  benchmarks 


Figure  1 7:  Results  and  Performance,  Measures  Tier 
Table  9  presents  examples  of  information  which  can  be  used  as  evidence  of  each  measure. 


Table  9:  Evidence  and  Instruments  for  Results  and  Performance 


Measure:  Quality  of  Business  Plan  and  Annual  Report 

Evidence 

Source/  Instruments 

•  DRDC  Business  Plan  and  Annual  Report 

•  Assessment  of  the  degree  to 
which  these  documents 
support  superior  (Report  on 
Plans  and  Priorities  (RPP), 
Departmental  Performance 
Report  (DPR))  or  external 
(transparency)  requirements 

Measure:  Risk-based  evaluation  plans  (reviewed  regularly)  and  follow-up 

Evidence 

Source/  Instruments 

•  Evidence  of  RDEC  decisions  on  past,  current  and  future 
requests  to  CRS  for  program  evaluations 

•  Program  evaluation 
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Measure:  Staff  and  client  survey  results 

Evidence 

Source/  Instruments 

•  Positive  survey  results 

•  Improved  results  over  time 

•  Staff  evaluate  managers’  performance  positively 

•  Public  Servant  Employee 
Survey 

•  Defence  Ethics  Survey 

•  DRDC  Client/  Partner 
Survey 

•  Annual  MAE  Managers 
Survey 

•  360-degree  feedback  on 
managers’  performance 

Measure:  Progress  in  strengthening  financial  and  program  results,  and  performance 
measurement 

Evidence 

Source/  Instruments 

•  Ongoing  tracking  of  available  measures  from  sources 
such  as  CPME;  e.g.:  ongoing  performance  against  project 
milestones;  expenditures  against  plans;  etc. 

•  Peer  review  files 

•  CPME 

•  Y ear-to-year  comparison  of 
PGAs 

Measure:  Performance  against  external  benchmarks 

Evidence 

Source/  Instruments 

•  Peer  reviews  of  scientific  programs 

•  Potentially,  evaluations  or  audits  of  corporate  services 
activities 

•  Environmental  scanning 

•  Peer  reviews 

•  Program  evaluations  and 
project  audits 

DRDC  Atlantic  CR  2009-135 


37 


Measure:  Corporate  monitoring  and  review  of  performance 

Evidence 

Source/  Instruments 

•  Coordinated  TPIRUST  groups 

•  Successful  horizontal  initiatives 

•  Quality  of  collaborative  relationships 

•  Program  Activity  Architecture 

•  Quality  of  evaluations,  including  focus  on  value  for 
money  and  program  improvement 

•  Neutrality  of  evaluation  function  (governance  and 
resources) 

•  Evaluation  coverage  of  the  organization’s  programs 

•  Extent  to  which  clear  performance  expectations  in  the 
Report  on  Plans  and  Priorities  (RPP)  are  tracked  and 
reported  on  in  the  Departmental  Performance  Report 
(DPR),  on  an  MRRS  basis 

•  Extent  to  which  the  Departmental  Performance  Report 
(DPR)  integrates  credible  performance  information 
(beyond  activities  and  outputs  to  results) 

•  Extent  to  which  the  Departmental  Performance  Report 
(DPR)  sets  performance  in  context,  is  balanced  and 
reports  on  lessons  learned 

•  PAA  reports 

•  Annual  MAP  Managers 
Survey 

•  Program  evaluation  of 
DRDPO  activities 

4.4  Learning,  Innovation  and  Change  Management 

Learning,  Innovation  and  Change  Management:  Expectations 

DRDC  fosters  creativity,  innovation  and  change,  promotes  organizational 
learning,  values  corporate  knowledge,  and  learns  from  its  performance. 

The  DRDC  expectations  statement  has  been  adjusted  to  better  reflect  the  high  level  of  creativity 
necessary  in  an  R&D  environment;  however  it  still  closely  reflects  that  of  the  TB  MAP,  shown 
below: 

The  department  manages  through  continuous  innovation  and  transformation, 
promotes  organizational  learning,  values  corporate  knowledge,  and  learns  from 
its  performance. 


Figure  18,  below,  depicts  the  Expectations  Statement  for  Learning,  Innovation  and 
Change  Management  in  the  DRDC  MAP. 
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Learning,  Innovation  and  Change 
Management 

DRDC  fosters  creativity,  innovation,  and 
change,  promotes  organizational 
learning,  values  corporate  knowledge, 
and  learns  from  its  performance 


Figure  18:  Learning,  Innovation  and  Change  Management,  Expectations  Tier 


Learning,  Innovation  and  Change  Management:  Indicators 

The  TB  MAP  employs  five  indieators  for  Learning,  Innovation  and  Change  Management,  shown 
below,  along  with  their  DRDC  MAP  counterparts: 

Strategic  organizational  learning,  a  capacity  to  anticipate  and  adjust  to  change,  and  a  disposition  to 

transformation:  The  ‘organizational  learning’  component  of  this  statement  has  been  highlighted  in 
the  indicator  DRDC  is  a  learning  organization  with  strategic  vision]  reference  to  change  and 
transformation  is  captured  elsewhere  in  the  indicators. 

A  culture  of  innovation:  This  indicator  has  been  coupled  with  concepts  of  change  and  transformation 
to  produce  the  DRDC  indicator  Organizational  culture  anticipates  change,  values  innovation,  and 
supports  transformation. 

Performance  as  a  guide  to  change:  This  indicator  has  been  re-worded  for  clarity  as  DRDC  examines 
its  own  performance  to  identify  need  for  change. 

Delegations  as  an  instrument  of  empowerment:  This  indictor  has  been  re-worded  to  Managers  are 
empowered  through  delegation  to  better  reflect  the  DRDC  management  structure. 

Corporate  knowledge  and  memory  captured  and  managed  as  strategic  resources:  This  indicator  has 
been  re-worded  as  Organizational  knowledge  and  memory  captured  and  managed  as  strategic 
resource  to  better  reflect  DRDC’s  status  as  an  Agency. 

Pigure  19,  below,  depicts  the  Indicators  for  Learning,  Innovation  and  Change 
Management  in  the  DRDC  MAP. 
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Learnii^,  Innovation  and  Change 
Management 

DKDCisa  learning  organization  with  strategic 
vision 

Organizational  cultiire  anticipates  chan^  values 
innovation,  and  supports  tiansfoimation 
DRDC  examines  its  own  performance  to  identify 
need  for  change 

Managers  are  enpowered  through  delegation 

Organizational  knowledge  and  memory  cultured 
and  mana^d  as  strategic  resources 


Figure  19:  Learning,  Innovation  and  Change  Management,  Indicators  Tier 


Learning,  Innovation  and  Change  Management:  Measures 

The  TB  MAP  employs  four  measures  for  Learning,  Innovation  and  Change  Management,  shown 
below,  along  with  their  DRDC  MAP  counterparts 

Progress  in  improving  organizational  learning  and  knowledge  management  practices:  this  measure 
has  been  divided  into  two  separate  measures  to  indicate  the  importance  of  each:  Progress  and 
investment  in  organizational  learning  and  Progress  in  knowledge  management  practices. 

Investments  in  organizational  learning:  has  been  incorporated  into  the  measure  Progress  and 
investment  in  organizational  learning. 

Stakeholder/staff  perceptions  of  organizational  adaptability,  change  and  innovation:  This  indicator 
can  be  directly  applied  to  the  DRDC  MAP  as  Stakeholder/staff perceptions  of  organizational 
adaptability,  change  and  innovation. 

Performance  measurement  used  to  improve  organizational  results:  this  measure  has  been  re -worded 
as  Linkages  between  organizational  performance  and  organizational  development  initiatives. 


Pigure  20,  below,  depicts  the  Measures  for  Learning,  Innovation  and  Change 
Management  in  the  DRDC  MAP. 
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Learning,  Innovation  and  Change 
Management 

Progress  and  investment  in  oiganizalional  learning 
Progress  in  kno'wledge  management  practices 
Stakeholder/staff perceptions  of  organizational 
ad^tability,  change  and  irmovation 
linkages  between  organizational  performance  and 
organizational  development  initiatives 

Figure  20:  Learning,  Innovation  and  Change  Management,  Measures  Tier 


Table  10  presents  examples  of  information  which  can  be  used  as  evidence  of  each  measure. 


Table  10:  Evidence  and  Instruments  for  Learning,  Innovation  and  Change  Management 


Measure:  Progress  and  investment  in  organizational  learning 

Evidence 

Source/  Instruments 

•  Change  management  plans 

•  PeopleSoft®  provides  information  on  training  and 
development  activities 

•  FMAS  provides  information  on  training  and  development 
direct  investment 

•  Corporate  initiatives  (such  as  EXPEDITION  09  1.4) 
provide  a  record  of  organizational  intent  and  commitment 

•  A  DRDC  MAE  survey  would  address  employees’ 
perceptions  of  the  effectiveness  of  organizational  learning 
initiatives  and  practices 

•  learning  strategy  that  is  integrated  into  HR/Business 
Planning  and  aligned  with  departmental  business 
priorities  and  management  improvement  objectives  of 
government 

•  An  organizational  learning  strategy  incorporating 
regularly  reviewed  learning  objectives,  opportunities  and 
requirements; 

•  A  strategy  to  determine  organizational  knowledge  needs 
as  well  as  to  capture,  manage  and  apply  organizational 
knowledge  to  shape  action  and  improve  results. 

•  PEP 

•  PMA 

•  Employee  survey 

•  Internal  Communications 
plan 

•  Change  management 
initiatives 
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Measure:  Progress  in  knowledge  management  practices 

Evidence 

Source/  Instruments 

•  Employee  participation  in  mentoring  program 

•  Employee  satisfaction  with  mentoring  program 

•  CPME  will  provide  information  on  investment  and 
progress  on  research  in  knowledge  management 

•  Assessment  of  operational  application  of  knowledge 
management  will  require  a  variety  of  instruments, 
including: 

♦  Review  of  CANDID  over  time 

♦  Review  of  EGAs 

♦  DRDC  MAE  survey  questions 

•  Employee  survey 

Measure:  Stakeholder/staff perceptions  of  organizational  adaptability,  change  and  innovation 

Evidence 

Source/  Instruments 

•  Employee  engagement 

•  Client/partner  surveys 

•  Staff  surveys,  either 
directly  as  DRDC  MAE 
surveys  or  potentially,  the 
Public  Service  employee 
Survey  if  appropriate 
questions  are  asked 

Measure:  Linkages  between  organizational  performance  and  organizational  development 
initiatives 

Evidence 

Source/  Instruments 

•  Analysis  of  DRDC  Business  Plans,  EGAs  and 
EXPEDITION  Ox  work  plans  to  establish  strength  of 
linkages. 

•  Eevel  of  organizational  engagement  on  change 
management 

•  Assignment  of  responsibilities  and  support  to  change 
management  practices 

•  Business  plans 

•  EGAs 

•  Expedition  XX  work  plan 

•  MAE  assessments 
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5  The  Core  Elements 


There  are  six  core  elements  of  the  DRDC  MAP :  Policy  and  Programs',  People,  Service',  Risk 
Management,  Stewardship;  and  Accountability.  These  six  core  elements  apply  to  everyone  in  the 
organization,  but  are  especially  relevant  to  line  managers,  whose  activities  determine  the  success  of 
DRDC  relative  to  MAP  measures. 

The  DRDC  MAP  core  elements  are  designed  to  parallel  the  TB  MAP  core  elements;  as  such,  the  TB 
MAP,  and  related  TB  documents  and  policies,  form  the  foundation  of  the  three  tiers  of  the  DRDC 
core  elements.  Additionally,  in  an  effort  to  increase  the  validity  of  our  operational  pilot  surveys,  the 
six  core  elements  also  build  on  academic  theory. 

5.1  Policy  and  Programs 

Mobilizing  Science  and  Technology  to  Canada’s  Advantage 

In  2007,  The  Government  of  Canada  released  a  guiding  document  for  Departments  and  Agencies 
involved  in  Science  and  Technology  called  “Mobilizing  Science  and  Technology  to  Canada’s 
Advantage”.  This  is  intended  to  be  a  strategic  document,  outlining  how  to  achieve  the  goals  -or 
Advantages-  set  in  the  2006  economic  plan  “Advantage  Canada”  by  setting  a  multi-year  S&T 
agenda  for  government.  The  recent  world  economic  crisis  will  likely  cause  a  revision  of  this 
economic  plan,  but  it  is  unclear  how  it  would  affect  S&T  related  goals. 

The  three  S&T  relevant  goals  are  identified  in  the  Mobilizing  Strategy  as:  Entrepreneurial 
Advantage;  Knowledge  Advantage;  and  People  Advantage.  These  three  Advantages  form  Canada’s 
Science  and  Technology  Pramework,  depicted  in  Pigure  21,  and  are  further  guided  by  four  core 
principles:  Promoting  world-class  excellence;  focusing  on  priorities;  encouraging  partnerships;  and 
enhancing  accountability. 


Industry  Canada,  Mobilizing  Science  and  Technology  to  Canada ’s  Advantage,  2007,  available  from:  : 
http://www.ic. gc.ca/eic/site/icl.nsf/eng/h  00231.html. 
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The  Klence  ind  Technology  Frameworfc 
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Chapter?; 

Entrepreneurial  Advantage 
Canada  mustlranslata 
IcnOiWiedga  into  practical 
application!  to  improw  ctr 
V!«allk!  walrwaa^  and  well-being. 


Giceernmentacdons  v<1ll  be  gulc^  by  tour  core  principles: 

■  Proraodng  Vibdd-class  excellence 

■  Foiushg  on  pilcrltles 

•  Encouraging  partnerships 

■  Enharclng  accountability 


Figure  21:  Canada ’s  Science  and  Technology  Framework^^ 


Chapter  5; 

People  A. dvanti^e 
Canua  must  lu  ba»  cf 
kr>ov/l»d9»  worbsrs  by  doivde^na 
arttracthg.ard  ivtanng  tha  highly 
sidikd  p»plft  vw  ntiad  to  Ichriw  in 
lh»  mMem  c^cbd  Kcoomy. 


Chapter  4: 

Knowledge  Adva  ntage 
Canada  mustbuild  upon  our 
r«4«rch  and  onginaarin^ 
strangthsy  ganarato  nv»  idoasand 
inrp^vTticns,  and  achiovo 
d  Kcol  I  d  n  CO  by  g  b  ba  I  s1a>d  a  rds. 


As  recently  as  2006,  Canada  was  identified  as  lagging  significantly  behind  other  OECD  countries 
for  intensity  of  investment  in  R&D.  Three  indicators  of  R&D  investment  intensity  (as  ratios  of 
GDP)  are  Gross  Expenditure  on  Research  and  Development  (GERD),  Business  Expenditure  on 
Research  and  Development  (BERD),  and  Higher  Education  Expenditure  on  Research  and 
Development  (HERD).  The  HERD  is  considered  a  surrogate  measure  of  public  expenditure  on 
R&D.  Of  these  three  indicators,  Canada  ranked  below  average  for  an  OECD  country  in  GERD  and 
BERD,  but  is  considered  a  leader  in  HERD.  Government  is  now  looking  to  improve  its  BERD  and 
GERD  standing  while  maintaining  its  HERD  status  by  pursuing  strategies  to  increase  R&D 
expenditure  through  supporting  entrepreneurship,  R&D  commercialization,  creating  a  taxation  and 
regulation  environment  that  encourages  R&D  activities,  and  offering  additional  funding  to  S&T 
students.^® 


Exploring  the  Core  Principles 

The  Science  and  Technology  Framework  is  guided  by  four  core  priorities.  This  section  will  explore 
how  these  principles  interact  with  DRDC’s  organizational  goals,  as  articulated  in  the  DRDC  2008 
Annual  Report. 


Industry  Canada,  2007,  p  12. 

Eleanor  Fast,  Mobilizing  Science  and  Technology:  The  New  Federal  Strategy,  Library  of  Parliament,  Science  and 
Technology  Division,  December  2007. 
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Promoting  world-class  excellence:  This  principle  aligns  perfectly  with  the  DRDC  vision  of 
becoming  a  world  leader  in  Science  and  Technology  for  defence  and  security.  The  Canada  S&T 
Framework  offers  the  following  strategies  for  achieving  world  class  excellence: 

Investing  in  funding,  scholarships,  and  internships  for  students  and  research  fellows; 

adjusting  taxation  and  regulation  to  attract  international-caliber  scientists  and  organizations  to 
Canada. 

While  it  is  beyond  the  ability  of  DRDC  to  affect  taxation  and  regulation  policy,  the  organization  can 
focus  on  attracting  and  retaining  new  and  experienced  scientists  to  the  organization.  Strategies 
related  to  these  priorities  are  explored  in  the  People  element  of  the  MAF. 

Focusing  on  priorities:  The  S&T  priorities  identified  in  the  Strategy  are:” 

Environmental  science  and  technologies; 

Natural  resources  and  energy; 

Health  and  related  life  sciences  and  technologies; 

Information  and  communications  technologies. 

DRDC’s  S&T  priorities  are  dictated  by  the  needs  and  demands  of  its  Departmental  clients  and 
partners,  namely  DND,  CF  and  Public  Safety  Canada.  Nonetheless,  while  conducting  its  S&T 
activities,  DRDC  needs  to  remain  conscious  of  Canada’s  broader  S&T  priorities  so  that  outcomes 
align  with  and  support  horizontal,  government-wide  initiatives. 

Encouraging  partnerships:  DRDC’s  mission  and  values  statements  both  highlight  a  ‘client  focus’ 
which  encourages  partnership  and  engagement  with  clients,  as  well  as  working  with  private  industry 
to  commercialize  R&D.  The  Canada  S&T  Framework  contains  recommendations  on  this  principle 
which  DRDC  can  employ,  such  as  diversifying  the  industry/business  base  of  R&D,  and  increasing 
the  commercialization  of  R&D  technology. 

Enhancing  accountability:  Canadians  demand  accountability  for  the  use  of  public  money  in  S&T. 
The  clearest  way  for  the  public  to  assess  accountability  is  by  determining:  a)  that  the  S&T  activity 
supports  government’s  stated  priorities;  b)  the  activity  has  clear,  measurable  success  targets;  and  c) 
the  activity  has  met  those  targets.  Accountability  focuses  on  results  for  Canadians. 

Currently,  there  is  a  lack  of  accountability  for  outcomes  in  S&T  in  Canada.^*  The  Government  of 
Canada  is  working  together  with  other  OECD  countries  to  develop  improved  indicators  and 
measures  of  S&T  impacts.” 

Generally  speaking,  as  an  organization,  DRDC  is  actively  seeking  to  enhance  Accountability 
mechanisms  through  the  development  and  application  of  the  Management  Accountability 
Framework. 


Industry  Canada,  2007,  p.  13. 
Eleanor  Fast,  p.  88. 

Industry  Canada,  2007,  p.  88. 
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What  is  Policy? 


Policy  making  in  the  context  of  S&T  can  be  generalized  into  two  categories:  Science  in  Policy  and 
Policy  in  Science.  The  first,  Science  in  Policy,  refers  to  any  number  of  policy  issues  in  government 
which  rely  on  scientific  or  technical  advice  for  informed  decision  making;  the  second.  Policy  in 
Science,  refers  to  policies  which  provide  the  management  framework  for  S&T  activities,  including 
how  R&D  programs  are  selected  and  evaluated. 

Whether  discussing  Science  in  Policy  or  Policy  in  Science,  policy  design,  analysis  and  evaluation  all 
follow  a  simple  policy  cycle.  The  cycle  includes: 

defining  the  problem; 

identifying  alternatives; 

assessing  alternatives; 

recommending  preferred  alternative; 

implementing  preferred  alternative; 

monitoring  implementation;  and, 

evaluating  results.'^' 

Easton  created  what  he  calls  a  ‘systems  framework’,  depicted  in  Figure  22,  outlining  the  factors 
which  influence  the  policy  cycle. In  Easton’s  model,  policy  is  simultaneously  a  dependent  and  an 
independent  variable. As  a  dependent  variable,  policy  is  influenced  by  such  inputs  as  political 
culture,  interest  groups,  social  ideology  and  values,  environmental  concerns,  and  so  on.  As  an 
independent  variable,  policy  impacts  outputs  such  as  regulation  and  legislation,  and  through  a 
feedback  loop,  influences  the  input  factors. 


Sheila  Jasanoff  referencing  Harvey  Brooks,  “Rationalizing  Politics”  in  The  Fifth  Branch:  Science  Advisors  as 
Policymakers.  Cambridge:  Harvard  University  Press,  1994,  p.  5. 

Paul  Brown,  Dalhousie  University,  private  communication. 

David  Easton,  “An  Approach  to  the  Analysis  of  Political  Systems,”  World  Politics,  9:3,  1957,  p.  384. 

Richard  Simeon,  “Studying  Public  Policy”,  Canadian  Journal  of  Political  Science  9:4,  1976,  p.  556. 
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Figure  22:  Easton ’s  System  Framework:  A  Public  Policy  Model 

In  the  DRDC  context,  much  of  the  policy  work  is  carried  out  by  scientists  and  technologists  through 
S&T  programs.  While  this  may  not  be  the  traditional  paradigm  of  policy  making,  the  advice  and 
reports  which  arise  from  these  programs  are,  in  fact,  policy  options. 


At  DRDC,  various  units  are  responsible  for  developing  Policy  in  Science: 

Director  S&T  (Policy)  staff  lead  the  development  of  S&T  Policy  for  DRDC  and  as  necessary  lead 
or  support  policy  development  for  DND/  CF  and  OGDs; 

CORA  staff  conduct  operational  research  and  analysis  which  supports  development  of  policies  for 
DND,  such  as  military  staffing  levels  or  stores  policies,  or  tacnotes  (tactical  directives); 

DGRDCS  staff  develop  DRDC  corporate  policies,  as  functional  authorities  for  financial, 
corporate,  and  HR; 

Staff  at  local  DRDC  centres  may  develop  policies  supporting  local  operations. 

Science  in  Policy  comes  out  of  the  scientific  R&D  work  done  through  DRDC  programs,  including 
the  Research  Technology  and  Analysis  (RTA)  Program,  Technology  Demonstration  Program 
(TDP),  Applied  Research  Program  (ARP),  and  Defence  Industrial  Research  Program.  Each  program 
is  comprised  of  a  number  of  different,  independent  projects.  Projects  with  common,  or 
interdependent,  interests  are  coordinated  through  Thrust  Advisory  Groups  (TAG). 

The  central  problem  of  Science  in  Policy  comes  out  of  Principal-Agent  theory. This  theory, 
borrowed  from  economics,  is  predicated  on  the  relationship  between  two  actors  -the  principal  and 
the  agent.  The  principal  does  not  have  the  appropriate  resources  (such  as  scientific  knowledge)  to 
achieve  its  goals,  and  so  must  use  the  resources  it  does  have  (such  as  money)  to  contract  an  agent 
who  possesses  the  required  resources  to  achieve  the  goals  on  its  behalf.®^  The  principals  in  this  case 
are  DND  /  CF  and  Public  Safety  Canada,  while  the  agents  are  the  scientists  of  DRDC. 


David  H.  Guston,  “Principal-agent  theory  and  the  structure  of  science  policy,  revisited:  ‘science  in  policy’  and  the  US 
Report  on  Carcinogens’'  Science  and  Public  Policy,  30:5  October  2003,  p.  347. 

Dietmar  Braun  and  David  H.  Guston,  “Principal-agent  theory  and  research  policy:  an  introduction,”  Science  and  Public 
Policy,  30:5  October  2003,  p.  303. 
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The  problem  that  arises  out  of  Principal-Agent  theory  is  that  of  information  asymmetry;  the 
principal  has  no  way  of  knowing  if  the  agent  is  providing  quality  outputs  because  the  principal,  by 
definition,  lacks  the  necessary  resources  to  judge  the  quality.  The  only  recourse  for  the  principal  is 
to  either  trust  the  agent  or  monitor  the  agent  in  an  effort  to  ensure  it  is  in  compliance  with  the 
contract.®'’  In  the  context  of  government  and  scientists,  the  need  for  Governmental  accountability  to 
Parliament  for  spending  and  program  quality  requires  that  agents  be  monitored. 

Exacerbating  the  issue  of  information  asymmetry  is  the  tension  between  the  technocratic  and 
democratic  views  of  policymaking,  particularly  in  sciences.®’  Technocrats  believe  that  science  and 
technology  is  neutral  and  the  best  policy  options  will  be  generated  if  experts  are  allowed  to 
formulate  policy  based  on  their  exclusive  knowledge.  Democrats  believe  that  science  and 
technology  have  social  and  political  biases,  so  for  policy  to  be  successful,  it  must  incorporate  public 
and  non-scientiflc  values.®*  An  effective  agent-monitoring  framework  would  need  to  balance 
rational,  evidence-based  decision  making  with  democratic,  public  participation.  ®® 

Science  in  Policy  and  Policy  in  Science 

The  central  problem  of  Policy  in  Science  is  one  of  lack  of  consistency  in  program  evaluation,  that  is 
a  lack  of  a  policy  framework  which  would  allow  one  to  answer  basic  questions  such  as  ‘does 
funding  and  delivering  on  this  project  further  the  goals  of  government?’  and  ‘did  this  project  meet 
its  goals?’.  The  lack  of  a  framework  also  creates  an  inability  to  compare  project  outcomes  from 
different  scientific  fields.’®  Development  and  application  of  a  policy  framework  that  addresses  these 
issues  would  also  function  as  a  monitoring  mechanism  for  assessing  accountability  in  the 
contributions  of  science  to  policy  making.  An  appropriate  monitoring  mechanism  can  provide  a 
double-barrelled  solution  by  addressing  the  issues  of  both  Science  in  Policy  and  Policy  in  Science. 

S&T  activity  outcomes  have  often  been  evaluated  at  the  Program  level,  according  to  overly  broad 
strategic  goals.’'  This  approach  makes  it  difficult  to  determine  the  success  of  any  one  individual 
project  within  a  program,  or  if  that  project  has  made  good  use  of  public  funding. 

One  promising  approach  to  measuring  and  evaluating  S&T  programs  is  to  borrow  from  investment 
management  concepts.  Science  and  technology  outcomes  are  assessed  from  a  ‘bottom-up’ 
perspective.  Each  project  is  evaluated  for  its  outcomes  or  ‘defined  investment  goal’  and  how  those 
outcomes  support  the  program  or  ‘portfolio’  goals.  Accountability  in  this  context  includes  having  a 
‘balanced  portfolio’  of  both  high  risk  and  low  risk  S&T  projects.”  In  the  vacuum  left  by  the  current 


Dietmar  Braun  and  David  H.  Guston,  p.  305. 

Sheila  Jassanoff,  p.  15. 

Philip  L.  Bereano,  “Reflections  of  a  Participant-Observer:  The  Technocratic/Democratic  Contradiction  in  the  Practice  of 
Technology  Assessment,”  Technological  Forecasting  and  Social  Change,  54:1997,  pp.  163-175. 

Melanie  DuPuis  and  Brian  J.  Gareau,  “Neoliberal  Knowledge:  the  Decline  of  Technocracy  and  the  Weakening  of  the 
Montreal  Protocol,”  Social  Science  Quarterly,  89:5  December  2008,  p.  1214. 

Irwin  Feller,  “Mapping  the  frontiers  of  evaluation  of  public-sector  R&D  programs,”  Science  and  Public  Policy, iA'.  10 
December  2007,  p.683. 

’*  Brian  A.  Jackson,  “Federal  R&D:  Shaping  the  National  Investment  Portfolio”  in  Shaping  Science  and  Technology 
Policy:  The  Next  Generation  of  Research,  David  H  Guston  and  Daniel  Sarewitz,  eds,  Madison:  University  of  Wisconsin 
Press,  2006,  pp.  33-54. 

’’Brian  A.  Jackson,  p.  34. 
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absence  of  an  accountability  approach  to  S&T/^  utilizing  this  goal-driven,  bottom-up  approach  to 
accountability  seems  prudent. 


Jackson  describes  portfolio  management: 

“The  central  challenge  in  portfolio  design  is  the  selection  of  investments 
so  their  component  goals  will  combine  appropriately  to  achieve  the  desired 
portfolio  goal.  Without  clearly  articulated  goals,  it  is  impossible  to  assess 
rationally  how  individual  R&D  investments  fit  as  components  of  program 
or  portfolio  goals.”^"* 

Notice  that  while  the  portfolio  concept  is  borrowed  from  the  financial  sector,  there  is  no  attempt  to 
attach  monetary  value  to  outcomes.  Instead,  the  outcomes  are  valued  according  to  whether  or  not 
they  have  achieved  their  goals.  By  articulating  the  desired  outcome  in  ‘units’,  we  can  select  the 
metric  which  is  appropriate  for  the  outcome;  the  specific  unit  metric  will  depend  on  the  goal.  By 
measuring  success  according  to  achievement  of  goals,  we  can  compare  success  rates  of  projects 
which  cannot  be  directly  compared  (ex:  workforce  development  and  knowledge  expansion). 

A  project  really  only  fails  when  it  does  not  meet  its  goal;  so  success  can  only  be  assessed  if  the  goal 
is  articulated.  Generally,  if  R&D  seeks  to  introduce  new  knowledge,  then  any  project  which 
contributes  to  scientific  knowledge  can  be  considered  a  success,  even  if  it  is  not  necessarily  ‘applied 
science’. 

Jackson  gives  five  generalized  goals,  each  with  different  desired  outcomes: 

Expansion  of  the  body  of  knowledge; 

Monetary  returns  via  economic  growth; 

Mission  directed  needs  for  R&D  results; 

Workforce  development  and  education;  and. 

Maintenance  of  national  scientific  and  technical  infrastructure  and  capacity. 

These  goals  can  be  aligned  with  the  Advantage  Goals  of  the  S&T  Strategy,  as  shown  in  Table  11. 


Eleanor  Fast,  p.2,  and  Industry  Canada,  p.  88. 
Brian  A.  Jackson,  p.  38. 
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Table  11:  Alignment  of  S&T  Strategy  with  Portfolio  Management 


S&T  Strategy"" 

Portfolio  Management 

Entrepreneurial  Advantage 

•  Monetary  returns  via  economic 
growth 

•  Mission  directed  needs  for  R&D 
results 

Knowledge  Advantage 

•  Expansion  of  the  body  of 
knowledge 

People  Advantage 

•  Workforce  development  and 
education 

•  Maintenance  of  national  scientific 
and  technical  infrastructure  and 
capacity 

The  ‘roll  up’  nature  of  the  portfolio  paradigm  also  fits  well  with  the  roll  up  nature  of  an  Agency 
level  MAT. 


Because  an  analysis  based  on  individual  investment  goals  begins  from  the 
bottom  up,  evaluation  at  different  levels  of  aggregation  becomes  more 
straight  forward.  . . .  (T)he  program  performance  of  any  office  is  simply  a 
roll-up  of  the  performance  of  its  individual  projects  and  assessment  of 
whether  the  projects  were  selected  appropriately.’*^ 

Policy  and  Programs:  Expectations 

The  Expectations  Tier  of  the  DRDC  MAE  was  developed  in  Cycle  1,  based  on  the  TB  MAE  and 
consultation  with  key  DRDC  personnel.  Figure  23  depicts  the  Expectations  Statement  for  Policy 
and  Programs  in  the  DRDC  MAE. 


Industry  Canada,  p.  13. 
Brian  A.  Jackson,  p.  46. 
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Policy  and  Programs 


DRDC  research  and  analytic  edacity  is 
developed  and  sustained  to  assure  high 
quality  policy  options,  program  design  and 
advice  to  DND/  CF,  Public  Safety  Canada, 
and  security  partners 


II 


Figure  23:  Policy  and  Programs,  Expectations  Tier 


Policy  and  Programs  :  Indicators 

Policy  capacity  can  be  defined  as  “the  ability  to  marshal  the  necessary  resources  to  make  intelligent 
collective  choices  about  and  set  strategic  directions  for  the  allocation  of  scarce  resources  to  public 
ends”’^  or  more  simply,  the  ability  to  consistently  create  and  implement  policy  which  meets  its 
intended  objectives. 

Policy  capacity  is  dependent  on  having  a  critical  mass  of  human  capital  trained  in  policy 
development,  implementation  and  evaluation,  as  well  as  a  policy  framework  or  process  in  which 
these  analysts  do  their  job,  and  a  culture  which  supports  good  policy  making.  Since  policy  capacity 
is  dependent  on  these  mutable  factors,  it  will  naturally  vary  over  time  and  place. 

The  Indicators  Level  for  Policy  and  Programs  reflects  the  three  key  indicators  needed  for  policy 
capacity:  people,  a  framework,  and  supportive  culture.  The  indicators  closely  resemble  those  of  the 
TB  MAP,  but  are  tailored  for  clarity  and  to  reflect  the  DRDC  environment  as  well  as  the  level  of 
DRDC  management  utilizing  the  MAP. 

Sustained  analytic  capacity  and  culture  of  consultation,  review  and  challenge:  This  indicator  remains 
unchanged  in  the  DRDC  Policy  and  Programs  indicators  tier. 

Results  focused  policy  and  program  agendas  linked  to  government’s  horizontal  priorities.  This 

indicator  has  been  revised  as  Results  focused  policy  and  program  agendas  aligned  with 
government’s  horizontal  priorities.  Because  DRDC  operates  as  a  Special  Operating  Agency,  it  is 
more  applicable  to  describe  the  Agency  agenda  as  aligned  with  that  of  government  as  opposed  to 
linked',  this  word  choice  better  reflects  that  the  work  carried  out  by  DRDC  is  targeted  to  internal 
clients  yet  indicates  that  the  outputs  of  that  work  advance  the  wider  Government  agenda.  It  is 
important  to  note  that  aligning  with  ‘horizontal  priorities’  “ensures  that  individual  policies  build  on 


Martin  Painter  and  Jon  Pierre,  “Unpacking  Policy  Capacity:  Issues  and  Themes,”  in  Challenges  to  State  Policy  Capacity, 
Martin  Painter  and  Jon  Pierre,  eds.  New  York:  Palgrave  Macmillan,  2005,  p.  2. 

Martin  Painter  and  Jon  Pierre,  p.  3. 
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each  other,  to  the  extent  possible,  and  minimises  inconsistencies  in  the  case  of  conflicting  policy 
goals...  [sjtrengthening  the  ‘whole-of-govemment’  perspective.”™ 

Citizen  Engagement.  This  indicator  needs  to  be  replaced  by  something  which  acknowledges  the 
need  to  consider  how  projects  and  programs  impact  citizens,  and  that  the  impacts  of  S&T  are  not 
neutral  in  society.  Social  Evaluation  of  Impacts  is  proposed.  In  some  cases  this  may  include 
engagement  of  citizens  to  determine  the  nature  of  impacts  of  S&T,  both  positive  and  negative,  on 
selected  groups. 

While  technocrats  might  be  more  efficient  in  their  processes. ..a 
fundamental  hallmark  of  democracy  is  the  realization  that  efficiency  may 
sometimes  be  compromised  because  of  our  devotion  to  multiple  values.^® 

Confidence  of  the  Minister  and  the  centre:  This  indicator  has  been  revised  as  Confidence  of  the 
ADM  (S&T),  clients  and  partners,  as  these  groups  are  the  direct  recipients  of  DRDC  policy, 
programs,  and  advice. 

Figure  24,  below,  depicts  the  Indicators  for  Policy  and  Programs  in  the  DRDC  MAF. 


Policy  and  Programs 

Sustained  analytic  edacity  andcultureof 
consultation,  review  and  challenge 

Results  focused  policy  and  program  agendas  aligned 
with  government's  horizontal  priorities 

Social  Evaluation  of  Infracts 

Confidence  of  the  ADM  (S&T),  clients  and  partners 


Figure  24:  Policy  and  Programs,  Indicators  Tier 


Policy  and  Programs:  Measures 

Measures  for  the  Policy  and  Programs  element  are  based  on  the  TB  MAF  Policy  and  Programs 
Measures  tier,  as  well  as  Round  VI  of  the  MAF  review  and  the  DND  2006  MAF  review.*' 

Confidence  of  the  minister  and  the  PCO  in  the  quality  of  policy  options  and  advice:  This  measure 
has  been  revised  and  shortened  to  simply  Quality  of  Policies  and  Programs.  The  confidence  of  the 
minister  and  PCO  is  not  relevant  to  the  DRDC  organizational  structure,  and  its  equivalent, 

OECD,  Government  Coherence:  The  Role  of  the  Centre  of  Government,  October  2000,  available  from: 
www.oecd.org/dataoecd/42/52/33981740.doc. 

*"  Philip  L.  Bereano,  p.  167. 

**  Treasury  Board  of  Canada  Secretariat,  MAF  Assessment  -  DND  2006,  available  from:  http://www.tbs-sct.gc.ca/maf- 
crg/assessments-evaluations/2Q06/dnd/dnd-eng.asD:  TBS,  Management  Accountability  Framework,  TBS,  MAF  Assessment 
-  Round  VI. 
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confidence  of  the  ADM  (S&T),  is  listed  above  as  an  Indicator,  so  will  not  be  repeated  here  as  a 
measure.  When  measuring  the  quality  of  policies  and  programs,  it  is  essential  to  consider  not  just 
outputs,  but  also  the  methodology  of  achieving  the  outputs  and  outcomes,  and  satisfaction  of  clients 
and  partners  with  the  methods  and  resulting  outcomes. 

Recruitment/  development/  succession  plans  for  policy  community:  The  HR  elements  of  this 
measure  have  been  more  succinctly  captured  in  the  statement  Comprehensive  Human  Resource 
Management  plan  for  the  policy  community.  Much  of  this  measure  is  addressed  as  part  of  the  People 
element  of  the  DRDC  MAP. 

Investments  in  policy  capacity/  analytic  tools:  This  measure  has  been  revised  as  Investments  in 
policy  capacity  and  research  tools  to  better  reflect  the  R&D  activities  of  DRDC. 

Figure  25,  below,  depicts  the  Measures  for  Policy  and  Programs  in  the  DRDC  MAP. 


Policy  and  Programs 

Quality  ofPolicies  and  Programs 

Conquehensive  Human  Resource  Management  plan 
firrthe  policy  community 

Investments  in  policy  edacity  and  research  tools 


Figure  25:  Policy  and  Programs,  Measures  Tier 
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Table  12  presents  examples  of  information  which  can  be  used  as  evidence  of  each  measure. 


Table  12:  Evidence  and  Instruments  for  Policy  and  Programs 


Measure:  Quality  of  Policies  and  Programs 

Evidence 

Source/  Instruments 

•  Clear  linkage  to  achievement  of  government 

•  Program/  Project  audit 

priorities 

•  Results  of  peer  reviews 

•  Reliability  of  information 

•  Number  of  ATI  requests  granted/ 

•  Peer  review/  challenge 

denied 

•  Realistic  timelines 

•  Social  Impact  Assessment 

•  Evidence  based  explanation  of  all  options  and 

checklist 

justification  of  preferred  options 

•  Environmental  Impact  Assessment 

•  Documented  risk  mitigation  strategies 

checklist 

•  Evidence  of  environmental  impact  assessment 

•  Client/partner  satisfaction  survey 

•  Evidence  of  social  impact  assessment 

•  Appropriate  stakeholder  consultations 

•  Client/partner  satisfaction 

•  Evaluation  of  results  with  feedback  mechanism 

Measure:  Comprehensive  HRM plan  for  the  policy  community 

Evidence 

Source/  Instruments 

•  Succession  Plan  in  place  for  Science  and 
Technology  personnel 

•  Policy  capacity  assessment  at  the 

Organizational  level 

•  Targets  for  recruiting  and  retaining  new  and 
experienced  scientists  and  technologists  in 
current  and  emerging  fields. 

•  Participation  in  co-operative  education 
programs  and  summer  student  internships 

•  Marketing  the  Agency  as  a  choice  employer  for 
scientists  and  technologists 

•  Audit  of  succession  plan 

♦  Existence  of  Plan 

♦  Implementation  of  Plan 

♦  Evaluation  for  results 

•  Audit  of  recruitment  and  retainment 
demographics 

•  Managers  survey 

•  Exist  survey  for  students  employees 

•  Effectiveness  of  student 
employment  programs 
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Measure:  Investments  in  policy  capacity  and  research  tools 

Evidence 

Source/  Instruments 

•  Maintaining  equipment  and  infrastmcture 

•  Partnering  with  academic  institutions 

•  Partnering  with  the  R&D  community 

•  Investing  in  new  technologies 

•  Funding  professional  development  and/or 
continuing  education 

•  Manager  survey 

•  Audit  of  CPME 

•  Evidence  of  Research  Collaboration 
through  CPME  or  manager  surveys 

•  Audit  of  professional  development 
programs 

•  Audit  of  professional  development 
programs 

•  Equipment  maintenance,  repair  and 
replacement  logs 

•  Percentage  of  secondment  requests 
approved 

5.2  People 

In  December  2005,  the  Government  of  Canada  fully  implemented  the  Public  Service  Modernization 
Act  (PSMA).  The  Act  was  created  to  improve  and  reform  the  Public  Service  in  order  to  meet  the 
demands  of  the  21®*  Century.  Factors  such  as  new  technology  and  globalization  had  lead  to  increased 
service  demands  and  heightened  expectations  from  citizens;  a  shift  was  occurring  in  labour  relations, 
leading  to  the  need  for  a  more  collaborative  labour  relations  model;  and  the  appointments  process 
had  become  cumbersome  and  ineffectual,  necessitating  the  creation  of  a  new,  streamlined  hiring 
system  which  could  function  more  effectively  while  protecting  the  core  principle  of  merit-based 
hiring.*^ 

The  PSMA  has  four  components,  which  work  together  to  enable  the  creation  of  a  new,  modem 
public  service. 

The  Public  Service  Employment  Act  (PSEA)*^ 

empowers  organizations  and  managers  by  encouraging  the  delegation  of  appointment  authorities 
from  the  PSC  to  deputy  heads; 

gives  deputy  heads  the  flexibility  to  tailor  their  staffing  system  to  meet  their  organizational  needs. 
They  also  have  greater  accountability  for  human  resources  management; 

provides  a  framework  for  staffing  that  is  simple,  flexible,  modem  and  more  responsive; 

provides  managers  with  the  flexibility  to  hire  the  right  people,  when  and  where  they  are  needed; 

Integrates  human  resources  and  business  planning;  and. 


Senator  Sharon  Carstairs,  Move  Second  Reading  of  Public  Service  Modernization  Act,  speech,  June  5,  2003. 

PSMA  Resource  Centre,  Key  Messages:  Public  Service  Employment  Act,  available  from:  httri://www.psagencv- 
agencefp.gc.ca/arc/hrmm-mgrh/psma-lmfp/centre/products-produits/messages  e.asp#PSEA.  (accessed  13  August  2009) 
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eliminates  the  appeals  process;  the  Public  Service  Staffing  Tribunal  now  deals  with  complaints 
related  to  internal  appointments  and  lay-offs. 

The  Public  Service  Labour  Relations  Act  (PSLRA):^"^ 

requires  all  deputy  heads  in  the  core  public  administration  to  establish  an  informal  conflict 
management  system  (ICMS)  and  communicate  its  existence  to  employees; 

requires  all  deputy  heads  to  establish  a  labour-management  consultation  committee; 

requires  the  negotiation  of  essential  services  agreements  between  employers  and  bargaining 
agents; 

enables  co-development  of  workplace  improvements; 

enhances  conciliation  (e.g.  through  the  appointment  of  a  public  interest  commission); 
provides  for  negotiated  essential  services  agreements; 

establishes  the  new  Public  Service  Labour  Relations  Board,  with  a  mandate  to  provide 
compensation  research  and  analysis,  as  well  as  adjudication  and  mediation  services; 

establishes  more  comprehensive  provisions  regarding  unfair  labour  practices; 

creates  more  comprehensive  grievance  and  adjudication  mechanisms;  and, 

requires  a  secret  ballot  before  a  strike. 

The  Financial  Administration  Act  (FAA)\  amendments  put  “direct  responsibility  for  certain  aspects 
of  human  resources  management  (such  as  determining  learning  and  developmental  requirements, 
providing  awards  and  setting  standards  of  discipline)  in  the  hands  of  deputy  heads.  With  this  new 
responsibility,  deputy  heads  are  required  to  submit  an  annual  report  to  Parliament  on  the  application 
of  the  human  resources  management  provisions  of  the  Act.”^^ 

The  Canadian  Centre  for  Management  Development  Act  (CCMDA):  Coordinate  and  integrate 
learning,  training,  and  development  activities  under  a  single  entity,  the  Canada  School  of  Public 
Service. 

Two  other  Legislative  drivers  impacting  Human  Resource  Management  in  the  Federal  Government 
are  the  Employment  Equity  Act  and  Official  Languages  Act. 

The  goal  of  the  Employment  Equity  Act  is: 

to  ensure  that  no  person  is  denied  employment  opportunities  and  benefits 
for  reasons  unrelated  to  ability.  It  requires  employers  to  correct 
disadvantages  in  the  workplace  experienced  by  members  of  the  four 


Canada  Public  Service  Agency,  Role  of  the  PSLRA  in  Public  Service  Modernization  Act  (PSMA)  Implementation  and  HR 
Modernization,  available  from:  http://www.psagencv-agencefp.gc.ca/arc/smp-gps/cm-gc/cmf-pslral  e.asp.  (accessed  13 
August  2009) 

PSMA  Resource  Centre,  Key  Messages:  Financial  Administration  Act,  available  from:  http://www.psagencv- 
agencefp.gc.ca/arc/hmm-mgrh/psma-lmfp/centre/products-produits/messages  e.asp#FAA.  (accessed  13  August  2009) 
PSMA  Resource  Centre,  Key  Messages:  Canadian  Centre  for  Management  Development  Act,  available  from: 
http://www.psagencv-agencefp.gc.ca/arc/hrmm-mgrh/psma-lmfp/centre/products-produits/messages  e.asp.  (accessed  13 
August  2009) 
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designated  groups:  women;  Aboriginal  peoples;  persons  with  disabilities; 
and  visible  minorities.*’ 

The  EEA  also  requires  that  employers  identify  and  eliminate  employment  barriers  for  designated 
group  members  and  make  reasonable  accommodations.  The  representation  of  persons  in  designated 
groups  should  reflect  their  representation  in  society. 

The  Official  Languages  Act  applies  to  all  federal  institutions.  The  Act  is  intended  to: 

ensure  respect  for  English  and  French  as  the  official  languages  of  Canada  and  ensure  equality  of 
status  and  equal  rights  and  privileges  as  to  their  use  in  all  federal  institutions,  in  particular  with 
respect  to  their  use  in  parliamentary  proceedings,  in  legislative  and  other  instruments,  in  the 
administration  of  justice,  in  communicating  with  or  providing  services  to  the  public  and  in 
carrying  out  the  work  of  federal  institutions; 

support  the  development  of  English  and  French  linguistic  minority  communities  and  generally 
advance  the  equality  of  status  and  use  of  the  English  and  French  languages  within  Canadian 
society;  and 

set  out  the  powers,  duties  and  functions  of  federal  institutions  with  respect  to  the  official 
languages  of  Canada.  ** 

The  TBS  has  coordinated  all  policies  related  to  Official  Eanguages  in  an  Official  Eanguages  Policy 
Framework,  available  on  its  website 

Staffing  Management  Accountability  Framework^^^  (SMAF)  translates  the  PSEA  into  a  more 
accessible  working  model.  It  meshes  with  the  MAE  to  provide  clearer  direction  on  the  People 
Element.  The  SMAF  is  comprised  of  eleven  elements: 

Delegation  of  Staffing; 

HR  Planning  and  Integration  with  Business  Planning; 

Organizational  HR  Support  Systems; 

Organizational  Accountability  for  Results; 

Flexibility  and  Efficiency; 

Merit; 

Non-partisanship; 

Representativeness; 

Access; 


Canadian  Human  Rights  Cormamion,  Frequently  Asked  Questions  about  Employment  Equity,  available  from: 
http://www.chrc-ccdp.ca/Dublications/ee  fag  ee-en.asp. 

Marie-Eve  Hudon,  The  Official  Languages  Act:  Understanding  its  Principles  and  Implementation,  Library  of 
Parliament,  October  2008,  available  from:  http ://www2 .pari. gc .ca/ Content/LOP/ResearchPublications/prb042 3-e.htm. 

Treasury  Board  of  Canada  Secretariat,  Official  Languages  Policy  Framework,  available  from:  http://www.tbs- 
sct.gc.ca/pol/doc-eng.aspx?id=12515&section=text. 

***  Public  Service  Commission,  Staffing  Management  Accountability  Framework,  available  from:  http://www.psc- 
cfp.gc.ca/plcv-pltq/frame-cadre/acco-resp/smaf-crgd-eng.htm. 
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Fairness;  and, 

Transparency. 

Other,  non-legislative  documents  relevant  to  Human  Resource  Management  in  the  Public  Service 
are  Treasury  Board's  Values  and  Ethics  Code  for  the  Public  Service‘^‘  and  Guidance  for  Deputy 
Ministers,  as  well  as  the  Privy  Council  Office  Annual  Report. 

Succession  planning  is  a  crucial  element  of  HR  management  and  is  vital  to  sustaining  capacity  in 
Policy  and  Programs. 

People:  Expectations 

The  Expectations  tier  of  the  DRDC  MAF  was  developed  in  Cycle  1,  based  on  the  TB  MAF  and 
consultation  with  key  DRDC  personnel.  The  expectations  elements  are  being  further  refined  as  the 
project  continues,  based  on  additional  research  and  an  understanding  of  the  interplay  among  the 
elements.  Figure  26  illustrates  the  revised  People  component  of  the  DRDC  MAF  Expectations  tier. 


People 


DRDC  has  the  people,  woik  environment, 
leadership  and  edacity  to  provide  the  best 
in  science  and  technology  for  defence  and 
security,  now  and  for  the  fiiture 


Figure  26:  People,  Expectations  Tier 


People:  Indicators 

Indicators  for  the  People  element  are  based  on  the  TB  MAF  People  Indicators  tier,  as  well  as  Round 
VI  of  the  MAF  review,  the  DND  2006  MAF  review,  and  the  eleven  elements  of  the  SMAF. 

The  TB  People  Indicators  tier  has  eight  individual  elements: 

Reflective  of  Canada 


**  TBS,  Values  and  Ethics  Code  for  the  Public  Service. 

Privy  Council  Office,  Guidance  for  Deputy  Ministers,  available  from:  http://www.pco- 
bcp.gc.ca/index.asp?lang=eng&page=information&sub=publications&doc=gdm-gsm/doc-eng.htm. 

Privy  Council  Office.  Annual  Reports  to  the  Prime  Minister  on  the  Public  Service  of  Canada,  available  from: 
http://www.pco-bcp.gc.ca/index.asp?lang=eng&page=clerk-greffier&sub=reports-rapports&doc=reports-rapports-eng.htm. 

Treasury  Board  of  Canada  Secretariat,  Succession  Planning  and  Management  Guide,  available  from:  http://www.tbs- 
sct.gc.ca/gui/sure-eng.asp. 
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Respectful  of  official  language  requirements 

Renewed/  sustained  capacity 

Supportive  workplace 

Employee  engagement 

Opportunities  to  grow 

Leadership  continuum 

Recognition,  rewards,  and  sanctions 

The  DRDC  Indicators  for  People  are  tailored  to  reflect  the  managers’  vernacular  and  the  DRDC 
environment. 

Reflective  of  Canada:  Renewed/  sustained  capacity:  These  indicators  speak  to  the  hiring  aspect  of 
Human  Resources.  As  an  equal  opportunity  employer,  the  Government  of  Canada  including  all 
federal  departments  and  agencies  should  have  a  workforce  which  approximates  the  demographics  of 
Canadians,  including  but  not  limited  to  the  four  designated  employment  equity  groups:  women,  the 
disabled,  visible  minorities  and  First  Nations  peoples.  It  is  important  that  the  workforce  maintain  the 
necessary  critical  mass  to  deliver  its  mandate,  and  that  employees  have  the  necessary  skills  and 
abilities  to  be  successful  in  the  delivery  of  programs  and  services.  These  indicators  are  combined 
into  a  more  succinct  indicator  Workforce  is  sufficient  in  numbers  and  abilities  while  being  reflective 
of  the  Canadian  population. 

Supportive  Workplace:  Respectful  of  official  language  requirements:  Recognition,  rewards. 

sanctions:  These  elements  are  captured  in  the  element  Workplace  is  a  supportive  and  respectful 
environment,  including  respect  for  official  language  requirements.  The  concept  of  sanctions  is 
addressed  in  the  Accountability  Element,  as  a  component  of  employee  performance  appraisals. 

Leadership  Continuum:  The  classic  Tannenbaum  and  Schmidt  model  of  the  Leadership  Continuum'*^ 
places  managers  on  a  scale  of  leadership  styles  ranging  from  ‘manager  centered’  to  ‘employee 
centered’.  In  the  parlance  of  the  TB,  the  leadership  continuum  has  two  meanings.  First,  it  refers  to 
all  supervisors  from  executives  to  functional  authorities,''®  all  of  whom  are  expected  to  demonstrate 
leadership  capabilities;  secondly,  it  also  refers  to  the  courses  available  to  help  these  individuals 
develop  leadership  capabilities.^^  This  element  is  captured  in  the  indicator  “All  levels  of  management 
demonstrate  effective  leadership” 

Employee  engagement:  Opportunities  to  grow:  These  elements  are  captured  in  the  indicator 
“Employees  are  engaged  in  their  work  and  have  opportunities  to  learn  and  grow” 

Figure  27,  below,  depicts  the  Indicators  for  People  in  the  DRDC  MAF. 


Robert  Tannenbaum  and  Warren  H.  Schmidt,  “How  to  Choose  a  Leadership  Pattern”,  Harvard  Business  Review,  May- 
June  1973,  reprint,  p.  164. 

Canada  Public  Service  Agency,  Key  Leadership  Competencies,  available  from:  http://publiservice.tbs- 
sct.gc.ca/tal/kcl/intro-eng.asp. 

Treasury  Board  of  Canada  Secretariat,  Directive  on  the  Administration  of  Leadership  Development  Programs,  available 
from:  http://www.tbs-sct.gc.ca/pol/doc- 

eng.aspx?evttoo=X&evttoc=X&section=text&id=12408&AspxAutoDetectCookieSupport=l#leadership. 
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People 

Workforce  is  sufScient  in  niimbeis  and  abilities 
^lile  reflecting  the  Canadian  population 

Workplace  is  a  supportive  and  respectful 
environment,  inchidingrespect  for  ofScial  langiia^ 
requirements 

All  levels  of  management  demonstrate  effective 
leadership 

Enqrloyees  are  eng^^d  in  their  work  and  have 
opportunities  to  learn  and  grow 


Figure  27:  People,  Indicators  Tier 
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People:  Measures 

Measures  for  the  People  element  are  based  on  the  TB  MAP  People  Measures  tier,  as  well  as  Round 
VI  of  the  MAP  review  and  the  DND  2006  MAP  review.^* 

Comprehensive  Pluman  Resources  development  plan  in  place,  including  leadership,  recruitment, 

retention,  succession,  learning,  Work/Life  Balance.  Official  Languages  and  Employment  Equity: 

This  measure  has  been  summarized  as  Comprehensive  Human  Resources  Strategic  Plan  in  place. 

Progress  against  Pluman  Resources  targets:  This  measure  has  been  applied  directly  to  the  DRDC 
MAP,  along  with  examples  of  three  core  HR  initiatives,  namely  quality  of  work  life,  official 
languages,  and  employment  equity.  The  measure  is  presented  as  Progress  against  HR  targets 
including  quality  of  work  life,  official  languages,  employment  equity. 

Progress  in  measuring/  improving  employee  engagement:  This  measure  has  been  applied  to  the 
DRDC  MAP,  with  a  small  style  adjustment,  as  Progress  in  measuring  and  improving  employee 
engagement. 

Quality  of  leadership:  This  measure  has  been  applied  directly  to  the  DRDC  MAP  as  Quality  of 
leadership. 

Quality  of  labour  relations:  This  measure  has  been  applied  directly  to  the  DRDC  MAP  as  Quality  of 
labour  relations. 

Pigure  28,  below,  depicts  the  Measures  for  People  in  the  DRDC  MAP. 


People 

Conqnehensive  HR  Strategic  Plan  in  place 

Progress  against  HR  tarots  includingqiiality  of  work 
life,  official  languages,  enq>loyee  equity 

Progress  in  measuring  &  in^rovingenq>loyee 
engagement 

Quality  of  leadeiship 

Quality  of  labour  relations 


II 


Figure  28:  People,  Measures  Tier 


There  are  a  number  of  information  sources  for  gathering  information  on  the  People  element.  A 
primary  resource  is  the  Public  Service  Employee  Survey,  conducted  by  the  Public  service 
Commission  (PSC)  every  three  years.  Utilizing  these  data  negates  the  need  to  ask  similar  or 
redundant  questions  in  a  DRDC  survey.  We  were  unable  to  access  these  data  as  a  part  of  the  MAP 
project;  however,  the  data  set  is  available  for  purchase. 


TBS,  MAP  Assessment  -  Round  VI;  TBS,  MAP  Assessment  -  DND  2006;  TBS,  Management  Accountability  Pramework. 
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Another  important  source  of  information  will  be  corporate  documentation  such  as  training  statistics 
(including  official  languages  training),  hiring  demographics,  promotion,  advancement  and 
secondment  demographics,  data  on  EAP  usage,  short  term  disability  and  workplace  safety. 
Departmental  Staffing  Accountability  Report  (DSAR)  may  be  a  useful  source  of  this  information. 

It  is  vital  that  DRDC  utilizes  a  centralized  Management  Information  System  (MIS),  such  as 
PeopleSoft®,  to  track  these  and  other  Human  Resource  Management  data  in  order  to  identify 
successes  and  failures  within  the  People  Element,  for  the  purpose  of  continuous  improvement. 

Table  13  presents  examples  of  information  which  can  be  used  as  evidence  of  each  measure. 


Table  13:  Evidence  and  Instruments  for  People 


Measure:  Comprehensive  HR  Strategic  Plan  in  place 

Evidence 

Source/  Instruments 

•  Centralized  MIS  for  tracking  HRM  data 

•  Succession  planning  model  in  place 

•  Number  of  supervisors  that  meet  the 
requirements  of  their  position 

•  DRDC  Human  Resources  Plan^^ 

•  PeopleSoft® 

Measure:  Progress  in  measuring  &  improving  employee  engagement 

Evidence 

Source/  Instruments 

•  Achievement  of  goals  in  employee  personal 
learning  plans 

•  Recognition  system  for  employee 
achievement 

•  Recognition  &  reward  program 
statistics 

•  Development  program  participation 
levels 

•  Personal  Learning  plan  statistics, 
especially  goal  achievement 

For  example,  Defence  Research  and  Development  Canada,  Human  Resources  Plan,  FY  2008-2009 . 
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Measure:  Progress  against  HR  targets  including  quality  of  work  life,  official  languages, 

employee  equity 

Evidence 

Source/  Instruments 

•  Established  targets  for  OE  requirements 

•  Established  targets  for  EE  representation  at 
different  employment  levels  for  designated 
groups  (women,  visible  minorities,  persons 
with  disabilities.  First  Nations) 

•  Established  targets  for  EE  representation  at 
different  employment  levels  for  non- 
designated,  but  marginalized  groups  (i.e. 
sexual  orientation,  age,  religion,  languages) 

•  Established  targets  for  WE  balance 

•  Existence  and  evaluation  of  programs  and 
policy  to  advance  towards  these  targets 

•  PeopleSoft® 

•  PS  Employee  Survey 

•  EAP  reports 

•  Progress  against  demographic  targets, 
employment  equity  staffing  activities, 
language  of  work  complaint  or 
discrimination  statistics 

Measure:  Quality  of  leadership 

Evidence 

Source/  Instruments 

•  Participation  in  secondments  and 
developmental  assignments 

•  Performance  Management  Program  for 
executive  level 

•  PS  Employee  Survey 

Measure:  Quality  of  labour  relations 

Evidence 

Source/  Instruments 

•  Eabour  Management  Consultation 
Committee  (EMCC)  in  place,  meets 
regularly 

•  Alternative  Dispute  Resolution  (ADR)  is 
in  place 

•  Unions  consulted  on  change  issues 

•  Management  understand  collective 
agreements  of  subordinates 

•  PS  Employment  Survey 

•  EMCC  and  EMRC  activities 

•  Alternative  Dispute  Resolution  statistics 

•  Classification  and  other  grievance 
statistics 
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5.3  Service 


The  TBS  document  Quality  and  Affordable  Service  for  Canadians:  Establishing  Service  Standards 
in  the  Federal  Government^^^  'was  implemented  in  1994.  The  service  standards  published  here  stress 
quality  and  affordable  services  at  the  citizen-public  servant  interface,  reflecting  the  fact  that  most 
government  departments  offer  services  directly  to  Canadians.  DRDC  is  an  exception  to  this,  since  its 
direct  services  are  delivered  to  DND,  other  government  departments  and  private  enterprises. 

For  these  reasons,  the  ‘Citizen-focused  Service’  element  of  the  TB  MAF  has  been  reduced  to  simply 
‘Service’  for  the  DRDC  MAF.  This  reduction  of  scope  is  consistent  with  the  TBS’  own  assessments 
of  DND  in  Round  V  of  the  TB  MAF: 


The  Department  of  National  Defence's  (DND's)  level  of  participation  in 
whole-of-govemment  service  solutions  and  the  extent  to  which 
public/client  views/needs  are  considered  when  developing  new  services 
were  not  assessed  as  they  were  not  applicable  in  DND's  (case).’®’ 

Most  service-oriented  literature  and  research  focuses  on  private  sector  organizations  delivering 
direct  customer  service;  however,  the  conclusions  and  best  practices  taken  from  these  studies  are 
relevant  to  the  public  sector,  and  are  readily  transferable  to  the  DRDC  service  model  in  which 
customers  are  replaced  by  client-partner  organizations. 


The  goal  of  service  delivery  is  to  satisfy  the  recipients  of  the  service.  Client  satisfaction  is  dependent 
not  only  on  service  quality  but  also  on  the  client’s  perceived  value  of  that  service,'**^  this  view  is 
echoed  in  the  very  title  of  the  TBS  service  standards  document  Quality  and  Affordable  Service  for 
Canadians.  There  is  evidence  that  high  satisfaction  levels  lead  to  increased  client  retention  and 
increased  regard  for  the  corporate  image.  This  relationship  can  be  summed  up  by  the  service 
equation  shown  in  Figure  29: 


Service 

Quality 


Perceived 

Client 

Corporate  Image 

Value 

Satisfaction 

♦ 

Client 

Retention 

Figure  29:  Service  Equation 


An  increased  regard  for  the  corporate  image,  as  well  as  the  retention  of  high  calibre  clients  and 
partners,  has  a  positive  impact  on  other  MAF  elements  as  well.  In  particular,  the  People  and  Policy 
and  Program  elements  call  for  strategic  recruitment  and  retention  of  new  and  experienced  scientists 
and  technologists,  as  well  as  positioning  the  organization  as  a  leader  in  the  S&T  community,  both  of 


Treasury  Board  of  Canada  Secretariat,  Quality  and  Affordable  Service  for  Canadians:  Establishing  Service  Standards  in 
the  federal  Government  -  An  overview,  available  from:  httri://www.tbs-sct.gc.ca/pubs  pol/opepubs/TB  D3/OOUA- 
eng.asp 

**’*  Treasury  Board  of  Canada  Secretariat,  MAF  Assessment-  Round  V,  available  from:  http://www.tbs-sct.gc.ca/maf- 
crg/assessments-evaluations/2007/assessments-evaluations-eng.asp. 

Hsin-Hui  Hu,  Jay  Kandampully,  and  Thanika  Devi  Juwaheer,  “Relationships  and  impacts  of  service  quality,  perceived 
value,  customer  satisfaction  and  image:  an  empirical  study”,  The  Service  Industries  Journal,  29:2  Feb  2009,  p.  121. 

Hsin-Hui  Hu,  Jay  Kandampully,  and  Thanika  Devi  Juwaheer,  p.  122. 

'"‘'based  on  Hsin-Hui  Hu,  Jay  Kandampully,  and  Thanika  Devi  Juwaheer’s  conceptual  model,  p.  1 13. 
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which  can  be  bolstered  by  a  strong  corporate  image  and  client  reputation.  By  focusing  on  increasing 
client  satisfaction  through  high  service  quality  and  high  perceived  value,  DRDC  service  delivery 
positively  impacts  these  three  MAP  elements. 

Service  Quality 

‘Service  quality’  is  a  construct  encapsulating  a  number  of  dimensions.  Most  construct  models,  as 
demonstrated  in  Ghobadian  et  ah,  can  be  aligned  to  group  the  various  dimensions  into  five  broad 
categories,  best  identified  by  Zeithaml  et  ah,  as  Tangibles,  Reliability,  Responsiveness,  Assurance, 
and  Empathy.'®'^  The  consumer  or  client  judges  the  quality  of  the  service  provided  based  on  his  or 
her  perception  of  these  dimensions,  which  may  not  necessarily  be  the  reality  of  the  dimension  in  the 
organization;  this  is  an  important  distinction.  For  example,  while  the  organization  may  in  fact  have 
cutting  edge  information  security,  the  client  may  not  perceive  this.  These  dimensions  are  designed  to 
apply  to  a  direct  client-provider  transaction;  however,  they  can  equally  be  applied  to  DRDC’s 
delivery  of  services  and  outputs  to  government  departments  and  private  enterprises. 

Tangibles  relates  to  the  quality  and  appearance  of  facilities,  personnel,  equipment,  and  other 
materials. In  the  case  of  DRDC  this  applies  to  the  appearance  and  quality  of  physical 
infrastructure  such  as  buildings,  laboratories  and  equipment,  the  appearance  of  personnel,  and  the 
appearance  and  quality  of  physical  outputs,  including  documents. 

Reliability  relates  to  the  organizations  ability  to  provide  services  and  outputs  dependably  and 
accurately.  In  the  DRDC  context,  this  includes  meeting  milestones  for  projects,  providing  project 
updates  on  time,  and  delivering  outputs  to  specification. 

Responsiveness  relates  to  the  willingness  of  personnel  to  meet  client  needs  and  respond  to  client 
requests  promptly'®^.  In  the  DRDC  context,  this  includes  correcting  or  addressing  problems 
promptly,  finding  mutually  satisfactory  solutions  to  issues,  and  keeping  clients  and  stakeholders 
proactively  informed  of  progress  and  problems. 

Assurance  relates  to  both  the  competence  and  courtesy  of  personnel,  and  the  credibility  and  security 
of  the  organization.  In  the  DRDC  context,  this  includes  having  highly  skilled  and  knowledgeable 
scientific  personnel  who  interact  with  clients  and  partners  in  a  courteous  manner,  all  personnel 
convey  trustworthiness  and  confidence  in  the  services  and  outputs  produced,  and  personnel  and 
tangibles  express  credibility  and  security. 

Empathy  can  be  thought  of  as  ‘individualized  attention’"'  and  encompasses  ease  of  access,  level  of 
communication,  and  understanding  the  customer.  In  the  DRDC  context,  this  includes  the  ease 
with  which  a  client  can  contact  the  appropriate  individual  in  the  organization,  the  quality  and 


Abby  Ghobadian,  Simon  Speller,  and  Matthew  Jones,  “Service  Quality:  Concepts  and  Models,”  International  Journal 
of  Quality  &  Reliability  Management,  11:9  1994,  pp.  53-54. 

Valerie  A  Zeithaml ,  A.  Parasuraman,  and  Leonard  L.  Berry,  Delivering  Quality  Service:  Balancing  Customer 
Perceptions  and  Expectations.  New  York:  Free  Press,  1990,  p.  26. 

Valerie  A  Zeithaml ,  A.  Parasuraman,  and  Leonard  L.  Berry,  p.  2 1 . 

Valerie  A  Zeithaml ,  A.  Parasuraman,  and  Leonard  L.  Berry,  p.  2 1 . 

Valerie  A  Zeithaml ,  A.  Parasuraman,  and  Leonard  L.  Berry,  p.  2 1. 

Valerie  A  Zeithaml ,  A.  Parasuraman,  and  Leonard  L.  Berry,  pp.  21-22. 

***  Valerie  A  Zeithaml ,  A.  Parasuraman,  and  Leonard  L.  Berry,  p.  26. 

"^Valerie  A  Zeithaml ,  A.  Parasuraman,  and  Leonard  L.  Berry,  p.  22. 
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quantity  of  communications  a  client  receives  or  engages  in,  and  the  organization’s  ability  and 
willingness  to  understand  issues  from  the  client  perspective. 

Core  requirements  to  achieving  high  service  quality  are:"^ 
developing  a  client  focus; 
empowering  front-line  staff; 
motivated  and  well-trained  staff;  and, 
a  clearly  defined  vision  of  service  quality. 

Service  quality  is  difficult  achieve  in  part  because  of  “three  features  unique  to  services:  intangibility, 
heterogeneity,  and  inseparability  of  production  and  consumption.”""*  Some  of  the  obstacles  to 
achieving  service  quality  include:"^ 

inability  to  perceive  problems  from  the  client’s  perspective; 

difficulty  in  identifying  the  origin  of  a  service  problem; 

resistance  of  service  personnel  or  systems  to  change;  and, 

failure  to  plan  for  contingencies. 

DRDC’s  core  services  are  the  delivery  of  Science  and  Technology  policy,  advice  and  products;"*^ 
the  service  quality  requirements  and  obstacles  described  above  are  readily  transferable  to  the  DRDC 
service  context. 

DRDC’s  success  in  the  S&T  domain  is  dependent  on  the  organization’s  S&T  competence,  and 
specifically  the  competence  of  S&T  staff  of  scientists,  engineers  and  technologists.  Its  program 
formulation  processes,  involving  R&D  Thrust  Advisory  Groups,  for  example,  are  clearly  client  (now 
partner)  focused.  Finally,  although  there  are  always  exceptions,  DRDC’s  professional  development 
policies,  and  career  management  for  defence  scientists,  provide  the  basis  for  the  well-trained  and 
motivated  staff  necessary  to  deliver  quality  services. 

Perceived  Value 

Various  authors  observe  that  evaluation  of  service  quality  is  difficult  since  the  quality  cannot  be 
measured  directly  (as,  for  example,  a  failure  rate  or  some  other  technical  measure)  but  must  be  based 
on  user  experience  or  expectation."’  It  can  be  difficult  for  an  organization  to  assess  or  anticipate  a 
client’s  perceived  value  of  a  service  because  each  individual  client  will  perceive  value  relative  to 
his/her  own  criteria."*  Generally,  clients’  perceived  value  of  a  service  will  be  based  on  a  “trade-off 
between  the  quality  or  benefits  [i.e.  utility]  they  perceive  in  product  relative  to  the  sacrifice  they 


Abby  Ghobadian,  Simon  Speller,  and  Matthew  Jones,  pp.  46-47. 

""'a.  Parasuraman,  Valerie  A  Zeithaml,  and  Leonard  L.  Berry,  “SERVQUAL:  A  Multi-Item  Scale  for  Measuring 
Consumer  Perceptions  of  Service  Quality,”  Journal  of  Retailing,  64: 1  Spring  1988,  p.  13. 

Abby  Ghobadian,  Simon  Speller,  and  Matthew  Jones,  pp.  46-47 
*’®Here,  ‘products’  is  used  generally,  ranging  from  reports  to  experimental  hardware,  and  including  in-service  support. 
*  For  example,  Ghobadian,  Speller  and  Jones;  or  Stephen  W.  Brown  and  Teresa  A.  Swartz,  “A  Gap  Analysis  of 
Professional  Service  Quality,”  The  Journal  of  Marketing,  53:2  April  1989,  pp.  92-98. 

*’*  Hsin-Hui  Hu,  Jay  Kandampully,  and  Thanika  Devi  Juwaheer,  p.  1 14. 
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perceived  by  paying  the  price.”"®  The  cost  or  ‘price’  of  a  service  can  include  money,  time, 
opportunity  costs  and  other  resources.  Because  the  organization  is  limited  in  its  ability  to  impact 
service  costs,  the  most  effective  method  for  influencing  perceived  value  is  by  maintaining  high 
service  quality. 

The  importance  of  perception  is  reinforced  in  an  earlier  study  by  Brown  and  Swartz,  which  explored 
professional  service  quality  in  primary  health  care  by  studying  client-professional  relationships 
between  doctors  and  patients.  Brown  and  Swartz  examined  the  gaps  between  client  expectations 
and  experience,  and  between  client  expectations  and  professional  perceptions  of  those  expectations. 
They  observe  the  need  either  to  align  service  provider  behaviours  and  expectations  with  those  of 
clients,  or  to  alter  client  expectations  and  experiences,  often  by  education  of  professional  capabilities 
and  services. 

Measuring  Service 

Measuring  service  quality  in  an  S&T  environment  is  more  complex  than  the  standards  illustrated  by 
Treasury  Board  Secretariat.  Setting  service  standards,  for  example,  may  be  straight  forward  in  a 
traditional,  client  centered  transaction,  such  as  Western  Economic  Diversification  Canada  (WEDC), 
where  “[rjesponse  to  initial  application  in  four  days”"'  is  presumably  a  reasonable  and  measurable 
service  standard,  given  a  90  percent  compliance  rate;  however,  the  same  is  not  the  case  for 
professional  services  (like  the  S&T  services  of  DRDC)  where  customer,  client  or  partner 
expectations  are  not  so  easily  quantifiable. 

Evaluation  of  service  quality  core  requirements  and  presence  of  obstacles:  Gather  information 
on  the  organization’s  performance  in  the  four  core  Service  quality  requirements,  as  well  as  areas 
where  obstacles  to  service  quality  may  exist,  through  an  internal  audit  of  core  requirements, 
obstacles,  service  standards  and  delivery.  The  MAF  survey  component  will  provide  a  valuable 
portion  of  this  evaluation. 

Perceived  Value  Gap  Analysis:  Perform  an  analysis  of  the  gap  between  client  expectation  and 
perception  of  service,  utilizing  a  gap  analysis  model,  such  as  Parasuraman  et  al,  shown  in  Figure  30. 


Hsin-Hui  Hu,  Jay  Kandampully,  and  Thanika  Devi  Juwaheer,  p.  1 14. 

Stephen  W.  Brown,  and  Teresa  A.  Swartz. 

™  TBS,  Quality  and  Affordable  Service  for  Canadians:  Establishing  Service  Standards  in  the  Federal  Government  -  An 
Overview. 
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Consumer 


Figure  30:  Quality  Gap  Analysis  Model‘^^ 


In  this  model  there  are  five  possible  gap  areas  where  client  expectations  of  service  may  differ  from 
their  perceptions  of  service.  By  analyzing  these  five  areas,  DRDC  can  minimalize  the  gap  between 
client  expectations  and  perceptions  of  service,  leading  to  increased  satisfaction.  The  five  gap  areas 
are: 

1)  Not  Knowing  Customers’  Service  Expectations; 

2)  Faulty  Service  Quality  Standards; 

3)  Service-  Performance  Gap; 

4)  Misrepresentation  of  Services;  and, 

5)  Gap  between  client  expectations  and  perceptions. 


Service:  Expectations 

Treasury  Board’s  Citizen-centred  Service  element. 


A.  Parasuraman,  Valerie  A  Zeithaml,  and  Leonard  L.  Berry,  “A  Conceptual  Model  of  Service  Quality  and  its 
Implications  for  Future  Research”,  Journal  of  Marketing,  49:4,  Autumn  1985,  p.  44. 
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Services  are  citizen-centred,  policies  and  programs  are  developed  from  the  "outside  in  ",  and 
partnerships  are  encouraged  and  effectively  managed. 

has  been  reduced  simply  to  ‘Service’  in  the  DRDC  MAP,  in  recognition  of  DRDC’s  predominantly 
internal  clients: 

S  &T  activities  are  client  and  partner-centred,  policies  and  programs  are  developed  to 
address  client  and  partner  needs,  and  partnerships  are  encouraged  and  effectively 
managed. 

The  Expectations  tier  of  the  DRDC  MAE  was  developed  in  Cycle  1,  based  on  the  TB  MAE  and 
consultation  with  key  DRDC  personnel.  Figure  3 1  illustrates  the  Service  component  of  the  DRDC 
MAE  Expectations  tier. 


Service 

S  &  T  activities  are  client  and  partner- 
centred,  policies  and  programs  are  developed 
to  address  client  and  partner  needs,  and 
partnerships  are  encouraged  and  efTectively 
managed. 


II 


Figure  31:  Service,  Expectations  Tier 


Service:  Indicators 

Treasury  Board  identifies  four  Indicators: 

Monitored,  continuously  improved  service  quality; 

Technology  options  fully  exploited; 

Empowered  front-line  deliverers;  and, 

Effective  relationships. 

Monitored,  continuously  improved  service  quality:  Defence  R&D  Canada’s  services  are  largely 
delivered  to  clients  internal  to  government,  but  service  quality  is  nonetheless  important.  Services 
include  the  delivery  of  policies  and  programs,  whose  quality  is  assessed  through  a  mixture  of 
internal  process  assessment  (e.g.:  performance  on  milestones  in  CPME,  the  Collaborative  program 
Management  Environment)  and  external  review  (e.g.:  peer  review  of  programs  or  client/partner 


TBS,  Management  Accountability  Framework. 
TBS,  Management  Accountability  Framework. 
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satisfaction  surveys).  The  suggested  DRDC  MAP  indicator  is  Monitored,  continuously  improved 
program  delivery. 


Technology  options  fully  exploited:  Treasury  Board  looks  to  technology  exploitation  for  the 
delivery  of  services  whereas  technology  options  are  often  the  deliverables  of  DRDC’ s  programs. 
Nonetheless,  technology  exploitation  does  have  a  role  in  delivery  of  DRDC’s  internal  services,  and 
in  delivery  of  policy  options  to  clients  and  partners  where,  for  example,  technology-based  decision 
support  tools  are  increasingly  used.  The  suggested  DRDC  MAP  indicator  is  Leading  advice  on 
technology  exploitation. 

Empowered  front-line  deliverers:  Defence  R&D  Canada  does  not  generally  offer  ‘front-line’  service 
to  citizens,  but  does  value  relationships  with  internal  clients  (generally  organizational  rather  than 
individual)  and  partners,  both  internal  and  external.  Defence  R&D  Canada’s  staff  are  expected  to 
engage  clients  and  partners,  to  build  relationships,  and  to  facilitate  new  partnerships.  The  suggested 
DRDC  MAP  indicator  is  Proactive  engagement  of  clients  and  partners. 

Effective  relationships:  Defence  R&D  Canada’s  S&T  programs  are  organized  using  a  mixed  in- 
house/extemal  delivery  model,  so  that  maintaining  effective  relationships  is  absolutely  necessary. 
Additionally,  DRDC  engages  internal  clients  and  partners  in  program  definition,  where  maintaining 
effective  relationships  is  again  critical.  The  suggested  DRDC  MAP  indicator  is  Effective 
relationships  with  clients  and  partners. 

It  would  be  tempting  to  collapse  the  last  two  indicators  into  one,  but  one  might  have  effective 
reactive  relationships  with  partners  without  ever  proactively  extending  the  partner  community. 

Pigure  32  depicts  the  Indicators  for  Service  in  the  DRDC  MAP. 


Monhoied,  continuously  improved  service  quality 
Leading  advice  on  technology  eiqiloitation 
Proactive  enga^ment  of  clients  and  partners 
Effective  relationships  with  clients  &  partners 


Figure  32:  Service,  Indicators  Tier 
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Service:  Measures 


The  Round  V  call  of  the  TB  MAP  opened  with  a  statement  for  Citizen-focused  Service  (2007): 

Services  are  designed  and  delivered  based  on  client  needs  and  expectations  and  continuously 
evolve  in  response  to  client  feedback. 

This  statement  is  readily  adaptable  to  the  DRDC  context,  and  has  been  considered  in  the 
development  of  Service  Measures  for  the  DRDC  MAP. 

Treasury  Board  initially  identified  six  measures,  with  the  fourth  having  been  overtaken  by  events:'^*’ 

Service  improvement  and  transformation  plans  in  place:  DRDC’ s  service  improvement  and 
transformation  plans  are  reflected  within  the  Defence  S&T  Strategy  for  S&T  activities  and  within 
the  EXPEDITION  09  organizational  development  project  for  the  DRDC  corporate  agenda.  The 
evolution  of  the  services  themselves  follows  the  business  planning  cycle  and  particularly 
client/partner  engagement  in  it.  The  suggested  DRDC  MAP  measure  is  Service  improvement  and 
transformation  planning  is  ongoing. 

Client  satisfaction  measured  annually:  The  Defence  S&T  Strategy  has  introduced  a  change  in  the 
DRDC  vernacular  from  clients  to  partners,  although  the  concept  of  client  satisfaction  is  applicable  to 
both  types  of  relationships.  It  has  been  DRDC’s  practice  to  measure  client  satisfaction,  although  it 
is  understood  that  corporate  re-organization  has  caused  suspension  of  the  formal  activity.  The 
suggested  DRDC  MAP  measure  is  Client/partner  satisfaction  is  measured  annually  against 
standards  or  expectations. 

Client  satisfaction  targets  and  results:  This  indicator  is  the  partner  of  the  previous  one,  addressing 
standards  rather  than  just  activity,  and  is  captured  by  the  measure  Client/partner  satisfaction  is 
measured  annually  against  standards  or  expectations. 

Progress  toward  Government  On-Eine  targets:  this  indicator  has  been  overtaken  by  events,  thus  it  is 
not  included  in  the  DRDC  MAP. 

Collaboration  with  other  governments  and  partners:  The  DRDC  business  model,  with  mixed  in- 
house  and  external  delivery  of  program,  client/partner  engagement  in  program  formulation,  and 
international  collaboration,  speaks  directly  to  this  element.  The  suggested  DRDC  MAP  measure  is 
Collaboration  is  key  to  delivering  DRDC  services. 

Information  for  citizens:  In  general,  DRDC  does  not  produce  information  for  citizens,  although  the 
Defence  Research  Reports  Database  is  an  obvious  exception.  The  suggested  DRDC  MAP  measure 
is  DRDC  makes  S&T  information  readily  available  to  stakeholders. 

Pigure  33  depicts  the  Measures  for  Service  in  the  DRDC  MAP. 


TBS,  MAP  Assessments  -  Round  V. 

126  Management  Accountability  Framework. 
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Service 

Service  iiiQtrDveiiient  and  transformation  planning  is 

on^ing  - - - 

CIient&  partner  satisfoction  is  measured  annually 
against  standards 

Collaboration  iskey  to  deliveringDREX^  services 

DRDC  makes  S&T  information  readily  available  to 
stakeholders 

Figure  33:  Service,  Measures  Tier 

Table  14  suggests  sources  and  instruments  for  the  proposed  Service  measures.  In  some  cases  survey 
instruments  are  proposed  for  data  collection  while  in  others,  corporate  publications  or  records 
should  provide  the  necessary  information.  In  many  cases,  assessment  of  performance  against 
particular  measures  will  require  multiple  instruments. 


Table  14:  Evidence  and  Instruments  for  Service 


Measure:  Client  &  partner  satisfaction  is  measured  annually  against  standards 

Evidence 

Source/  Instruments 

•  Client  satisfaction  survey  annually 

•  Open  communication  with  clients  and 
partners 

•  Client  feedback  is  utilized  to  refine  systems 
and  informs  future  projects 

•  Corporate  services  files 

•  Surveys  of  internal  clients  and  service 
providers 

•  Comparative  analysis 

•  DRDC  client  surveys  and  peer  reviews 

•  Survey  of  managers  addressing 
perceptions  of  client  satisfaction  and 
peer  review 

•  Client  surveys 

•  Overview  Group  records 

•  MAE  surveys  (DRDC  manager 
perceptions) 
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Measure:  Service  improvement  and  transformation  planning  is  ongoing 

Evidence 

Source/  Instruments 

•  Percentage  of  projects  which  are  implemented 
by  partners 

•  Integrated  System  for  tracking  service  targets 

•  Targets  based  on  client  survey  results 

•  Planning  for  contingencies 

•  Project  managers  are  empowered  to  be 
responsive  to  changes  in  client  needs 

•  EXPEDITION  XX  dashboard  and 
progress  reports 

•  DRDC  Annual  Report 

•  MAE  surveys  to  measure  cultural 
adoption 

Measure:  Collaboration  is  key  to  delivering  DRDC  services 

Evidence 

Source/  Instruments 

•  Policies  and  project  deliverables  are  developed 
with  a  client  focus 

•  Partners  are  regularly  given  project  updates 

•  Availability  of  DRDC-wide  SOPs 

•  Analysis  of  CORS  Working  Group 
minutes 

•  Provider  and  client-focused  surveys 

•  Survey  of  individual  collaborative 
activities 

•  Partner  and  client-focused  surveys 

•  MAE  surveys  (Collaboration) 

•  Annex  on  S&T  Collaboration 

Measure:  DRDC  makes  S&T  information  readily  available  to  stakeholders 

Evidence 

Source/  Instruments 

•  Acknowledgement  of  citizens  as  part  of 
client  base 

•  Transparency  and  availability  of  non- 
classified  information  and  reports 

•  DRDKIM  records  (CANDID,  etc.) 

•  DRDC  Annual  Report 

•  Partner  and  client-focused  surveys 
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5.4  Risk  Management 


Risk  is  an  inevitable  factor  in  most  decision  making  situations.  Risk  most  often  has  a  negative 
connotation  and  many  conventional  definitions  describe  risk  as  the  possibility  of  an  unwanted 
outcome,  or  failure  to  achieve  a  desired  outcome;'^’  however,  innovation  is  also  the  offspring  of 
risk,  and  an  unanticipated  outcome  can  actually  prove  to  be  highly  valuable.  This  tension  between 
risk  as  failure  and  risk  as  innovation  is  particularly  relevant  in  science  and  technology  fields.  An 
organizational  Risk  Management  Framework  can  help  guide  decision  makers  in  determining  the 
level  of  risk  acceptable  to  strategically  minimize  losses,  maximize  opportunities  (including  possible 
innovations)  and  prepare  for  uncertainty. 

The  TBS  Integrated  Risk  Management  Framework  defines  risk  as  “the  uncertainty  that  surrounds 
future  events  and  outcomes,  ft  is  the  expression  of  the  likelihood  and  impact  of  an  event  with  the 
potential  to  influence  the  achievement  of  an  organization’s  objectives.”'^®  This  definition  highlights 
the  notion  that  an  outcome  only  has  meaning  relative  to  how  it  affects  the  organization’s  ability  to 
achieve  its  goals. 

Risk  Management  is  defined  in  the  Framework  as  “a  systematic  approach  to  setting  the  best  course 
of  action  under  uncertainty  by  identifying,  assessing,  understanding,  acting  on  and  communicating 
risk  issues.”'^"  Comcover  Australia  employs  a  similar  definition  of  risk  management:  “[it  is]  the 
activities  and  actions  taken  to  ensure  that  an  organization  is  conscious  of  the  risks  it  faces,  makes 
informed  decisions  in  managing  these  risks,  and  identifies  and  harnesses  potential  opportunities.”'^' 
ft  is  vital  to  the  Framework  that  the  organization  has  a  Corporate  Risk  Profile,  a  document  which 
outlines  the  levels  of  risk  that  are  considered  acceptable  to  the  organization;  this  ‘risk  tolerance’  aids 
decsion  makers  in  determining  when  the  costs  associated  with  risks  are  acceptable.  A  sound  risk 
management  plan  is  also  a  key  component  in  maintaining  accountability  among  public  servants 
when  decision  making  is  required  during  critical  incidents. 

While  risk  management  is  a  process  utilized  to  consider  the  risks  associated  with  specific  scenarios, 
integrated  risk  management  takes  a  broader  view  of  risk  as  it  affects  the  organization  horizontally 
and  vertically.  The  TBS  defines  Integrated  Risk  Management  as 

a  continuous,  proactive  and  systematic  process  to  understand,  manage  and  communicate  risk 
from  an  organization-wide  perspective.  It  is  about  making  strategic  decisions  that  contribute 

to  the  achievement  of  an  organization ’s  overall  corporate  objectives. 


Comcover,  Risk  Management:  Better  Practice  Guide,  Australian  Government,  June  2008,  p.  5,  available  from: 
Available  from:  http ://www. finance . gov.au/ comcover/better-practice-guide.html . 

Comcover,  p.  5. 

Treasury  Board  of  Canada  Secretariat,  Integrated  Risk  Management  Framework,  April  2001,  available  from 
http://www.tbs-sct.gc.ca/pol/doc-eng.aspx?evttoc=X&id=12254&section=text. 

130  Integrated  Risk  Management  Framework. 

Comcover,  p.  5. 

™  Audit  Office  of  New  South  Wales,  Performance  Audit  Report:  Risk  Management  in  the  NSW  Public  Sector,  June  2002, 
available  from:  http://www.audit.nsw.gov.au/publications/reports/performance/2002/risk  manage/Risk-Contents.html. 

Peter  Aucoin  and  Ralph  Heintzman,  “The  dialectics  of  accountability  for  performance  in  public  management  reform,” 
International  Review  of  Administrative  Sciences,  66,  2000,  p.  48. 

TBS,  Integrated  Risk  Management  Framework. 
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The  DND  Integrated  Policy  on  Risk  Management  also  utilizes  this  definition. 

In  2004,  Chief  Review  Services  contracted  Deloitte  and  Touche  to  perform  a  baseline  study  of 
Integrated  Risk  Management  (IRM)  within  DND/CF.'^'’The  study  found  considerable  weaknesses  in 
the  Departments’  Corporate  IRM  approach,  consistent  with  TB  MAP  assessment  of  DND/CF  in  this 
element.  See  Table  15,  below. 


Table  15:  Comparison  of  IRM  Key  Elements  against  DND/CF  Practice^^^ 


IRM  Key  Element/  Characteristic 

DND/CF  Comparison 

Continuous,  dynamic  risk  identification  as 
early  warning 

Relatively  sporadic  and  annual  identification 

Possible  risk  events  proactively  identified 
before  occurrence 

Largely  reactive  to  risk  event  occurring 

Systematic  process  in  place 

Risks  considered  principally  as  they  relate  to 
business  planning 

Structured  analysis  of  likelihood  and  impact 

Mainly  intuitive  analyses,  although  pockets 
where  structure  is  used 

Everyone  identifies  risk 

Mostly  a  manager’s  responsibility  to  identify 
risk 

Organization-wide  process 

Process  not  yet  in  place 

Risk  managed  at  the  lowest  practical  level 

Risk  tolerances  often  not  known  or 
communicated;  therefore,  difficult  for  lower 
levels  to  manage  risks 

Risks  prioritized 

Unstructured  prioritization 

Reporting  of  prioritized  risks  upwards 

Reporting  partially  through  annual  business 
planning 

Mitigation  plans  commensurate  with  severity 
and  likelihood  of  risks 

Few  mitigation  plans  based  on  risk  assessment 

Open  communication  of  risks 

Limited  horizontal  communication 

A  number  of  recommendations  were  made  to  address  areas  of  weakness  in  the  DND/CF  Integrated 
Risk  Management  performance;  all  align  with  the  TBS  Integrated  Risk  Management  Framework. 
The  TBS  Integrated  Risk  Management  Framework,  in  turn,  appears  to  be  rooted  in  High  Reliability 
Organization  Theory  (HRO).  Developed  by  a  group  of  social  science  researchers  at  Berkley 
University  through  case  studies  of  organizations  with  low  accident  and  system  failure  rates,  the 
HRO  fundamental  principle  is  that  serious  accidents  and  failures  can  be  prevented  through  good 
organizational  design  and  management. 

In  response  to  the  Deloitte  and  Touche  study,  DND/CF  developed  two  risk  management  documents, 

138  139 

Integrated  Risk  Management  Policy  and  Integrated  Risk  Management  Guidelines.  These 


Department  of  National  Defence,  Integrated  Risk  Management  Policy,  2005,  available  from: 
http://vcds.mil.ca/dgsp/OOnative/rep-pub/ddm/riskman/IRJVlPol  e.pdf  (DWAN  only) 

Chief  Review  Services-  DND,  Baseline  Study:  Integrated  Risk  management  within  the  DND/CF,  1 000-6-4  (CRS) 
January  2004. 

Chief  Review  Services-  DND.  Baseline  Study:  Integrated  Risk  Management  within  the  DND/CF,  p.  III/IX. 

DND,  Integrated  Risk  Management  Policy,  2005. 
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documents  are  no  longer  available  through  the  DND  website.  DND  has  also  developed  a  Corporate 
Risk  Profde;  however,  it  has  not  yet  received  final  approval  for  publication. 

Table  16,  opposite,  compares  High  Resiliency  Organization  Theory  with  the  TBS  Integrated  Risk 
Management  Framework  and  DND  Integrated  Risk  Management  Policy  and  Guidelines.  As  a 
Special  Operating  Agency  of  DND/CF,  DRDC  would  develop  its  Risk  Management  Framework 
based  on  TBS  and  DND/CF  documents  and  practices. 

Overall,  HRO  provides  a  sound  theoretical  basis  for  the  development  of  the  DRDC  Risk 
Management  Framework,  with  a  few  caveats.  As  public  organizations  accountable  to  citizens. 
Government  departments  have  additional  responsibilities  in  terms  of  Risk  Management,  which  are 
not  specifically  addressed  in  HRO,  including  the  need  to  identify  and  consult  stakeholders 
throughout  the  RM  process,  and  the  need  to  find  a  balance  among  the  risks,  costs  and  benefits  of  a 
decision  or  initiative  to  achieve  an  outcome  with  a  net  social  benefit;  innovation  may  be  one  of  these 
outcomes. Another  drawback  of  the  HRO  approach  is  that  it  fails  to  take  into  account  the 
interdependency  of  organizations  through  networks  and  partnerships. 


Department  of  National  Defence,  Integrated  Risk  Management  Guidelines,  2005 . 
Private  Communication,  Karim  Dahel. 

TBS,  Integrated  Risk  Management  Framework,  p.  10. 
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Table  16:  Alignment  of  High  Resiliency  Organization  Theory  with  TBS  Integrated  Risk 
Management  Framework  and  DND  Integrated  Risk  Management  Policy  and  Guidelines 


Item 

High  Resiliency  Organization 
Theory 

TBS  Integrated  Risk 
Management  Framework^*^ 

DND  Integrated  Risk 
Management  Policy^**  and 
Guidelines'*^ 

Organizational 

Culture 

Organizational  culture  exists 
which  values  safety  and  risk 
management. 

‘Risk  Smart’  workforce 
supports  responsible  Risk 
Management. 

Risk  Management  must  be 
acknowledged  as  a 
foundation  of  management. 

Senior  Management 
Commitment 

Senior  management 
demonstrates  safety  as  a 
priority  in  the  organization. 

Senior  management  function 
as  Risk  Management 
champions 

Executive  guidance  and 
support  is  essential. 

Training 

Ongoing  Risk  Management 
and  safety  training  for  all 
members  is  crucial. 

Need  for  formal  training  in 
Risk  Management  principles, 
applications  and  tools. 

N/A 

Risk  Management 
Process 

Anticipation  and  Resilience: 
brainstorming  all  conceivable 
failures  in  order  to  prevent 
them,  plus  having  plans  in 
place  to  deal  with  them  should 
they  arise. 

Ongoing  internal  and  external 
environmental  scanning  to 
identify,  assess,  mitigate,  and 
exploit  risks. 

Risks  are  identified  through 
a  workshop  approach,  with 
a  brainstorming  technique. 
Identified  risks  are 
prioritized  and  risk 
responses  developed. 

Risk 

Communication 

There  is  a  communication 
mechanism  for  sharing 
information  about  risk  and 
lessons  learned  from  failures, 
near  failures  and  successful 

•  147 

projects. 

Reporting  on  performance  is 
necessary  to  identify  lessons 
learned,  best  practices,  and 
monitor  use  of  resources. 

Risks  can  be  documented  in 
a  risk  register  or  log.  A  best 
practices  repository  should 
be  developed.  External 
consultation  needs  to  be 
considered  within  the 
context  of  the  Access  to 
Information  Policy 

Integration  / 
Redundancy 

Redundant  mechanisms  are  in 
place  to  monitor  high-risk 
technologies.*''* 

Risk  Management  is 
integrated  within  existing 
structures  of  decision 
making,  reporting,  operations 
and  governance. 

Risk  management  is  an  on¬ 
going  activity,  embedded 
within  current  processes. 

Decentralization 

Risk  Management  is 
decentralized  so  that  safety 
and  risk  issues  are  dealt  with 
at  the  lowest  level  of 

149 

management. 

As  a  component  of  Risk 

Smart  Culture,  all  public 
servants  are  expected  to  be 
risk  aware  and  integrate  Risk 
Management  into  their 
existing  processes. 

Risk  Management  is 
everyone’s  responsibility; 
risks  are  shared  horizontally 
or  vertically  as  needed. 

Scott  D.  Sagan,  “Chapter  One:  The  Origins  of  Accidents”  in  The  Limits  of  Safety:  Organizations,  Accidents,  and 
Nuclear  Weapons,  Princeton:  Princeton  University  Press,  1993. 

TBS,  Integrated  Risk  Management  Framework. 

DND,  Integrated  Risk  Management  Policy. 

DND,  Integrated  Risk  Management  Guidelines. 

Aaron  Wildavsky,  as  referenced  in  Scott  D.  Sagan,  p.  16. 

This  mechanism  acts  as  a  feedback  loop  for  making  adjustments  to  policy  and  procedures  in  order  to  improve 
operational  safety  and  success. 

*'**  Communication  and  information  sharing  is  a  form  of  redundancy/  cross  check,  see  Scott  D.  Sagan,  p.  20. 

Strong  organizational  risk  management  culture,  including  training  and  continuous  learning,  allows  for  decentralization 
of  risk  decision  making  to  the  operational  level  while  assuring  that  responses  and  conclusions  will  be  consistent  with 
organizational  objectives,  see  Scott  D.  Sagan,  p.  24. 
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Risk  Management:  Expectations 

The  Expectations  Tier  of  the  DRDC  MAE  was  developed  in  Cycle  1,  based  on  the  TB  MAE  and 
consultation  with  key  DRDC  personnel.  Figure  34  illustrates  the  Risk  Management  component  of 
the  DRDC  MAE  Expectations  tier. 


Risk  Management 

DRDC  prepares  for,  manages  and  exploits 
risk  in  the  science  and  technology 
environment 


II 


Figure  34:  Risk  Management,  Expectations  Tier 


Risk  Management:  Indicators 

The  Treasury  Board  MAE  explains  the  Indictors  tier  as  descriptive  statements  of  the  Expectations 
tier  “meant  to  convey  the  breadth  and  meaning  of  the  expectations  ...  [to]  help  to  gauge  progress 
toward  those  objectives.”'^® 

The  TB  RM  Indicators  tier  has  four  elements;  the  DRDC  Indicators  for  Risk  Management  are  based 
on  those  of  the  TB  MAE,  with  Eligh  Resiliency  Organization  Theory  in  mind.  The  Indicators  are 
tailored  to  reflect  the  DRDC  environment  and  level  of  management  utilizing  the  MAE. 

Key  risks  identified  and  managed:  In  order  to  manage  risk  successfully,  the  organization  must  be 
aware  of  its  risk  areas,  risk  tolerances,  and  ability  to  mitigate  and  exploit  risk.  As  a  Special 
Operating  Agency,  DRDC  falls  under  the  purview  of  DND;  its  CEO  is  accountable  to  the  DM  of 
DND.  This  element  is  captured  in  the  indicator  Risk  is  managed  within  the  context  of  the  DND 
Corporate  Risk  Profile. 

Risk  lens  in  decision  making:  A  risk  lens  can  be  thought  of  as  a  permanent  filter  through  which  all 
decision  making  is  viewed;  this  is  done  by  incorporating  Risk  Management  into  all  existing 
planning  and  operational  processes  and  governance  and  organizational  structures.’^’  This  element  is 
captured  in  the  indicator  Risk  Management  is  integrated  with  planning  and  operations  structures 
throughout  all  levels  of  the  Agency. 

Risk  smart  culture:  The  TB  identifies  a  ‘Risk  Smart  Culture’  as  one  which  “has  the  capacity  and 
tools  to  be  innovative  while  recognizing  and  respecting  the  need  to  be  prudent  [.]”’^^  This  element  is 
captured  in  the  indicator  Capacity  exists  to  support  a  Risk  Management  Culture.  Capacity  in  this 


150  Management  Accountability  Framework. 

TBS,  Integrated  Risk  Management  Framework,  p.  5. 
*^^TBS,  Integrated  Risk  Management  Framework,  p.  5. 
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instance  refers  to  the  time,  tools,  training  and  support  needed  to  enable  risk  management  to  become 
integrated  throughout  the  agency  at  all  levels. 

Capacity  to  communicate  and  manage  risk  in  public  context.  This  element  is  captured  in  the 
indicators  Risk  Management  is  integrated  with  planning  and  operations  structures  throughout  all 
levels  of  the  Agency  together  with  Capacity  exists  to  support  a  Risk  Management  Culture. 

Figure  35,  below,  depicts  the  Indicators  for  Risk  management  in  the  DRDC  MAF. 


Risk  Management 

Risk  is  mana^d  within  the  context  ofthe  DND 
Coipoiate  Risk  Profile 

Risk  Management  is  integrated  with  plarmingand 
operations  structures  througlioiit  all  levels  ofthe 
A^ncy 

C^racity  exists  to  support  a  Risk  Mana^ment 
Culture 


II 


Figure  35:  Risk  Management,  Indicators  Tier 


Risk  Management:  Measures 

The  Measures  tier  of  the  TB  MAF  includes  four  measures  for  Risk  Management: 

Corporate  Risk  Profile,  reviewed  regularly; 

Tools,  training,  support  for  staff; 

Evidence  of  risk  considerations  in  strategic  planning;  and. 

Engagement  of  external  stakeholders  in  assessing  /  communicating  risks. 

In  a  MAF  assessment  of  select  departments  in  2006,  including  DND,  TB  used  the  following  as 
measures  of  risk: 

Eevel  of  engagement  of  the  senior  executive  in  corporate  risk  management  (e.g.  organizational 
focus,  senior  executive  accountabilities,  risk  mitigation  strategies.  Business  Continuity  Plans, 
Emergency  Preparedness  Plans); 

Currency  of  the  corporate  risk  assessment  and  profile; 

Extent  to  which  corporate  planning  is  informed  by  risk;  and. 

Risk  Mitigation  strategies  are  in  place  for  key  program  and  corporate  risks. 


TBS,  Management  Accountability  Framework. 
TBS,  MAF  Assessment  -  DND  2006. 
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Round  VI  of  the  TB  MAP  concerning  Risk  Management,  shown  below  in  Table  17,  refines  these 
measures  in  an  effort  to  align  them  with  the  forthcoming,  new  TBS  Risk  Management  Framework 
Policy,  set  to  be  introduced  in  2009. 

Table  17:  Risk  Management  in  the  Treasury  Board  MAF,  Round  VI 


The  executive  team  clearly  defines  the  corporate  context  and  practices  for  managing 
organizational  and  strategic  risks  proactively. 

Risk  is  a  factor  in  corporate  decision-making  processes  and  management 

Evidence: 

•  Senior  management  directs,  participates  in  and  provides  oversight  of  the 
organization’s  Risk  Management  approach 

•  Organization  is  implementing  its  Risk  Management  approach 

•  Risk  Management  explicitly  informs  corporate  decision-making,  planning  and 
reporting 

•  Quality  assurance  of  risk  information  and  continuous  improvement  in  Risk 
Management 


Based  on  these  inputs,  the  suggested  DRDC  Measures  tier  for  Risk  Management  attempts  to 
incorporate  the  Round  VI  evidence  items  with  the  TB  MAF  Measurements  tier  as  it  currently  stands. 
There  is  no  one-to-one  correspondence  between  the  Risk  Management  measures  of  the  TB  MAF  and 
those  of  the  DRDC  MAF. 

Figure  36,  below,  depicts  the  measures  for  Risk  Management  in  the  DRDC  MAF. 


Treasury  Board  of  Canada  Secretariat,  MAF  VI  Methodology  Changes  2008-2009:  Risk  Management,  2008,  available 
from:  http://www.tbs-sct.gc.ca/maf-crg/indicators-indicateurs/2008/risk-risaues/risk-risaues-eng.asp. 
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Risk  Management 

RiskMmia^Dient  is  integrated  with  decision  making, 
planning  and  lepoiting 

RiskMana^ment  tools,  training,  and  support  are  in 
place  for  staff 

Mana^ment  demonstrates  leadership  in  Risk 
Mana^ment 

RiskMana^ment  ^rproach  congrlimentsDND 
corporate  risk  profile. 


Engagement  of  external  stakeholders  in  assessing  / 
communicatingrisks 


Figure  36:  Risk  Management,  Measures  Tier 


Table  18  presents  examples  of  information  which  can  be  used  as  evidence  of  each  measure. 


Table  18:  Evidence  and  Instruments  for  Risk  Management 


Measure:  Risk  Management  is  integrated  with  decision  making,  planning  and  reporting 

Evidence 

Source/  Instruments 

•  A  systematic  process  is  in  place  for  managing  risks 
and  that  process  is  utilized  proactively  by  all 
employees 

•  Familiarity  with  the  DND  policy  and  guidelines  on 
Risk  Management 

•  All  standard  reports  include  a  section  on  Risk 
Management,  indicating  the  identification  and 
assessment  of  risks 

•  Risk  Management  tools,  such  as  impact  /likelihood 
matrix,  are  integrated  in  the  decision  making  process 

•  A  formal  recording  system  exists  to  track  risks  and 
reporting 

•  Lessons  learned  from  past  experiences  can  be  shared 

•  Evidence  of  risk 

management  processes  in 
project  documentation  and 
CPME 
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Measure:  Risk  Management  tools,  training,  and  support  are  in  place  for  staff 

Evidence 

Source/  Instruments 

•  Employees  receive  training  and  continuous  learning 
in  IRM,  in  a  conscious,  coordinated  fashion. 

•  Employees  are  aware  of  what  resources  are  available 
to  them  in  RM  (ex:  SWAT  analysis  tools) 

•  Risk  management  roles  and  responsibilities  are 
assigned 

•  IRM  is  linked  to  performance  reviews/  rewards 

•  PeopleSoft®  training 
statistics 

•  Risk  management  SOPs,  etc 

Measure:  Management  demonstrates  leadership  in  Risk  Management 

Evidence 

Source/  Instruments 

•  Management  actively  fosters  an  environment  where  it 
is  acceptable  to  not  have  all  the  answers. 

•  Innovation  is  encouraged  as  an  outcome  of  risk 
taking. 

•  A  common  language  exists  for  discussing  risk. 

•  Management  surveys 

Measure:  Risk  Management  approach  compliments  DND  corporate  risk  profile 

Evidence 

Source/  Instruments 

•  The  DND  corporate  risk  profde  is  available  for 
employees  to  review 

•  DRDC  risk  tolerance  level  is  explicit  and  understood 

•  Annual  reports  and  business  plans  are  aligned  with 
priorities  identified  in  the  Corporate  Risk  Profile 

•  The  Corporate  Risk  Profile  is  reviewed  regularly 

•  Use  of  best  practices 
repository 

•  DRDC  Corporate  Risk 
Profile 

•  DRDC  Business  Plan  and 
Annual  Report  alignment 
with  Corporate  Risk 
profile  (DND  and  DRDC) 
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Measure:  Engagement  of  external  stakeholders  in  assessing  /  communicating  risks 

Evidence 

Source/  Instruments 

•  Employees  are  aware  of  their  responsibilities  and 
limitations  in  terms  of  RM  (i.e.-  when  they  can 
address  an  issue  and  when  it  needs  to  be  sent 
upwards) 

•  Risk  information  is  shared  horizontally  across  the 
agency 

•  Lessons  learned  are  shared  within  the  Agency  and 
with  relevant  external  communities 

•  Review  of  TDP 
documentation  and 
associated  Senior  Review 
board  minutes 

5.5  Stewardship 

The  term  ‘stewardship’  is  defined  as  “the  careful  and  responsible  management  of  something 
entrusted  to  one's  care.”'^'^  In  the  public  sector,  government  departments  act  as  stewards  of  the 
resources  of  Canadians;  national  resources  include  money  (specifically  tax  revenue),  information, 
environmental/  natural  resources,  national  reputation,  people,  and  physical  assets. 

The  Stewardship  element  of  the  Treasury  Board  MAP  seeks  to  place  controls  on  how  government 
employees  manage  resources.  Similar  to  the  problem  discussed  in  the  Policy  and  Programs  element, 
the  central  issue  in  Stewardship  is  based  on  Agency  Theory.  This  theory,  borrowed  from  economics, 
is  predicated  on  the  relationship  between  two  actors  -the  principal  and  the  agent.  The  principal  is  the 
party  which  ‘owns’  the  resource,  while  the  agent  is  the  party  responsible  for  managing  the  resource. 
The  principals  in  this  situation  are  Canadian  taxpayers,  while  the  agents  are  the  government 
departments  and  employees.  Each  party  seeks  to  maximize  the  utility  of  the  resource.  When  the 
interests  of  the  principal  and  the  agent  coincide,  there  is  no  agency  problem;  however,  when  their 
interests  are  divergent,  an  agency  problem  exists.'^’  Because  the  agent  has  control  of  the  resources, 
s/he  can  utilize  the  resources  to  his/her  benefit,  resulting  in  agency  costs;  the  principal  can  reduce 
these  costs  by  imposing  internal  controls  on  the  agent,  such  as  audit  requirements,  reporting 
structures,  and  evaluations.'^* 

Controls  placed  on  agents  need  to  be  sufficient  to  curb  agency  costs  without  being  overly  costly  or 
cumbersome.  An  overly  rigid  control  regime  not  only  utilizes  excessive  resources  in  comparison  to 


Merriam- Webster  Online  Dictionary,  available  from:  http://www.merriam-webster.com/dictionary/stewardship. 
James  H.  Davis,  F.  David  Schoorman,  and  Lex  Donaldson,  “Toward  a  Stewardship  Theory  of  Management,” 
of  Management  Review,  22:1,  1997,  p.  22;  Dietmar  Braun  and  David  H.  Guston,  “Principal-agent  theory  and  research 
policy:  an  introduction,”  Science  and  Public  Policy,  30:5,  October  2003,  p.  22. 

James  H.  Davis,  F.  David  Schoorman,  and  Lex  Donaldson,  p.  22. 
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the  potential  for  abuse  of  resources,  it  can  serve  to  disengage  employees  and  build  resentment 
towards  the  organization,  leading  to  diminished  motivation,  and  loss  of  trust. 

An  alternative  theory  may  offer  valuable  information  to  help  enhance  trust  in  the  organization. 
Ironically  named  Stewardship  Theory,  it  seeks  to  explain  why  it  is  that  an  agent  will  sometimes  act 
in  the  interests  of  the  principal  even  when  to  do  so  appears  counter  to  the  agent’s  own  benefit.'®® 

Stewardship  Theory,  also  rooted  in  economic  theory,  describes  the  relationship  between  the 
principal  (Canadian  taxpayers)  and  the  stewards  (government  departments  and  employees).  Just  as 
in  Agency  Theory,  each  party  seeks  to  maximize  the  utility  of  resources  to  his/her  own  benefit.  The 
steward  has  control  of  the  principal’s  resources,  and  will  utilize  the  resources  to  maximize  his/her 
own  utility,  which  may  result  in  agency  costs  if  the  steward’s  interests  are  not  aligned  with  those  of 
the  principal.  Stewardship  Theory  postulates  that  the  steward  places  higher  value  on  cooperative 
behaviours  because  this  will  provide  greater  utility  than  self-serving  behaviours. The  principal  can 
encourage  cooperative  behaviours  by  granting  the  steward  a  degree  of  trust;  this  will  lead  to  intrinsic 
motivation  to  achieve  the  organization’s  goals,  based  on  the  steward’s  desire  for  higher  order  needs 
such  as  self-  determination,  achievement,  and  personal  growth. 

Successful  Stewardship  in  the  MAP  context  will  depend  on  finding  the  correct  balance  between 
imposing  internal  controls  and  granting  subordinates  an  appropriate  measure  of  trust  to  function 
autonomously.'®^ 

Stewardship:  Expectations 

The  Expectations  tier  of  the  DRDC  MAP  was  developed  in  Cycle  1,  based  on  the  TB  MAP  and 
consultation  with  key  DRDC  personnel.  Pigure  37  illustrates  the  Stewardship  component  of  the 
DRDC  MAP  Expectations  tier. 


A  [“ 

^■11 

II 

Stewardship  ^ 

The  DRDC  control  regime  (assets,  money, 
people,  services,  etc.)  is  integrated  and 
effective,  and  its  underlying  principles  are 
clear  to  all  staff. 

Figure  3  7:  Stewardship,  Expecations  Tier 


Jens  Grundei,  “Chapter  3:  Examining  the  Relationship  between  Trust  and  Control  in  Organizational  Design”  in 
Organization  Design:  the  Evolving  State  of  the  Art.  R.M.  Burton,  B.  Eriksen,  D.D.  Hakonsson,  C.C.  Snow,  eds.  Springer 
Information  and  Organization  Design  Series,  6,  2006,  p.  52;  James  H.  Davis,  F.  David  Schoorman,  and  Lex  Donaldson,  p. 
25. 

James  H.  Davis,  F.  David  Schoorman,  and  Lex  Donaldson,  p.  24. 

James  H.  Davis,  F.  David  Schoorman,  and  Lex  Donaldson,  p.  24. 

Jens  Grundei,  p.  52;  James  H.  Davis,  F.  David  Schoorman,  and  Lex  Donaldson,  p.  37. 

Jens  Grundei,  p.  49. 
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Stewardship:  Indicators 


The  DRDC  Stewardship  Indicators  Tier  has  been  revised  to  better  reflect  the  tone  of  the  overall 
DRDC  MAP  and  to  more  succinctly  capture  the  Stewardship  Indicators  of  the  TB  MAP  and  the 
Stewardship  Assessment  Methodology  Changes,  Rounds  IV  -VI.  Below  is  a  comparison  of  the  TB 
MAP  indicators  and  the  revised  DRDC  indicators  for  the  Stewardship  element. 

Management  systems  that  provide  relevant  information  and  early  warning  on  resources,  results  and 

controls:  This  indicator  has  been  re-worded  as  Management  systems  provide  relevant  information 
and  effective  feedback  on  resources,  results  and  controls.  Management  systems  include  information, 
technology,  financial,  asset,  security,  and  people  management  systems.  Project  management  has 
been  singled  out  as  a  separate  indicator  because  of  its  significance  in  the  DRDC  environment. 

Rigorous  audit/  evaluation  function:  This  indicator  has  been  re-worded  as  Established  internal  audit 
function.  Performance  evaluation  is  addressed  in  the  People  element. 

Punctional  specialists  as  partners:  This  indicator  has  been  folded  into  the  indicator  Management 
systems  provide  relevant  information  and  effective  feedback  on  resources,  results  and  controls. 
Issues  around  asset  management  and  procurement  are  specifically  addressed  in  the  measures  tier. 

Compliance  with  policies  regulations,  and  legislation:  This  indicator  remains  unchanged  from  the 
TB  MAP. 

Pigure  38,  below,  depicts  the  revised  Indicators  for  Stewardship  in  the  DRDC  MAP. 


Stewardship 

Management  systems  provide  relevant  infonnation^ 
and  effective  feedback  on  resources,  results  and 
controls 

Effective,  efficient  and  economical  project 
management 

Established  internal  audit  function 

Compliance  with  policies,  regulations,  and 
legislation 

Hll 

II 

Figure  38:  Stewardship,  Indicators  Tier 


Stewardship:  Measures 

Measures  for  the  Stewardship  element  are  based  on  the  TB  MAP  Stewardship  Measures  tier,  as 
well  as  Round  VI  of  the  MAP  review,  the  DND  2006  MAP  review,  and  results  of  the  Cycle  1  Pilot 
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Survey  As  such,  there  is  no  one-to-one  correspondence  between  the  Stewardship  measures  of  the 
TB  MAP  and  those  of  the  DRDC  MAP. 

Pigure  39,  below,  depicts  the  Measures  for  Stewardship  in  the  DRDC  MAP. 


Stewardship 

Reports  and  records  are  timely  and  accurate 
Expenditures  are  within  bud^ 


Resources  are  procured  and  used  sq^rpropiiatelyto 
achieve  ^als 

Assets  are  inventoried  and  strategically  mana^d 
Information  is  utilired  and  protected  ag^rpropriately 
Deliverables  are  timely  and  effective 


Figure  39:  Stewardship,  Measures  Tier 

Table  19  presents  examples  of  information  which  can  be  used  as  evidence  of  each  measure. 
Table  19:  Evidence  and  Instruments  for  Stewardship 


Measure:  Reports  and  records  are  timely  and  accurate 

Evidence 

Source/  Instruments 

•  Pinancial  forecast  performance 

•  Managers’  survey 

•  Audit 

•  Pinancial  forecast  performance 

•  Manager  survey 

Measure:  Expenditures  are  within  budget 

Evidence 

Source/  Instruments 

•  Pinancial  Management  &  Control 

•  PMAS  and  forecast  performance 

•  CPME  year-to-year  comparisons 

MAP  Assessment  -  Round  VI;  TBS,  MAP  Assessment  -  DND  2006;  TBS,  Management  Accountability 
Pramework;  Craig  O’Blenis  and  Warren  Nethercote,  ^  DRDC  Management  Accountability  Pramework:  Cycle  1  Pinal 
Report. 
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Measure:  Resources  are  procured  and  used  appropriately  to  achieve  goals 

Evidence 

Source/  Instruments 

•  Certified  procurement  specialists  at  each 
centre 

•  User  Iriendly  procurement  process 

•  Procurement  file  performance 

•  Number  of  specialists  at  each  centre 

•  Level  of  activity  at  each  centre 

•  Average  turnaround  of  requests 

•  Manager  survey 

Measure:  Assets  are  inventoried  and  strategically  managed 

Evidence 

Source/  Instruments 

•  Scheduled  inventory  assessment 

•  Designated  position  with  responsibility  for 
asset  management  in  each  unit 

•  Unused  or  underused  assets  are  shared 
among  units 

•  Asset  inventories 

•  Frequency  of  asset  sighting 

•  Write-offs  or  transfers 

Measure:  Information  is  utilized  and  protected  appropriately 

Evidence 

Source/  Instruments 

•  Security  infraction  statistics  (if  releasable) 

•  Protection  of  personnel  files 

•  Protection  of  classified  files/assets 

•  Audits 

Measure:  Deliverables  are  timely  and  effective 

Evidence 

Source/  Instruments 

•  Milestone  achievement 

•  Program  audits 

•  CPME 
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The  following  issues  were  noted  in  the  Cycle  1  report,  based  on  results  of  the  Stewardship  pilot 
survey  conducted  in  summer  2008. 

Financial  Management  and  Control:  Although  responses  were  generally  good,  there  were 
instances  of  ineffective  use  of  delegations  that  suggest  a  need  for  professional  development  of 
subordinates. 

Project  Management:  Although  project  management  practices  seem  to  be  present  in  DRDC,  there 
are  still  many  instances  where  a  project  management  culture  is  not  fully  established. 

Procurement:  There  seem  to  be  issues  around  procurement  processes  and  reporting.  The 
processes  may  be  a  consequence  of  the  FAA,  and  so  invariant,  but  improved  clarity  of  process 
and  reporting  of  progress  of  files  might  address  this  issue  indirectly. 

Asset  Management:  Capital  planning  appears  to  be  weak  among  level  3  managers. 

Information  Management:  Project  documentation,  and  financial  record  reconciliation  are  both 
candidates  for  improvement  among  level  3  managers. 

These  issues  were  revisited  by  Pilot  Survey  3  in  Cycle  2  to  determine  if  there  has  been  change  in 
these  areas.  Some  of  the  issues,  such  as  delegation  and  project  management,  were  addressed  in  part 
in  Pilot  Survey  2  of  Cycle  2  under  the  auspices  of  Risk  Management  and  Accountability. 

5.6  Accountability 

Accountability  has  become  a  cornerstone  of  good  governance  in  Canada,  but  what  does 
‘accountability’  in  the  government  context  really  mean?  The  wide  use  (and  misuse)  of  the  term 
‘accountability’  has  resulted  in  a  muddying  of  its  meaning. 

Romzek  and  Dubnick  describe  four  alternative  systems  of  accountability  within  the  public 
administration  context:  bureaucratic,  legal,  professional  and  political  accountability.'®’ Each  of  these 
systems  is  based  on  the  relative  impact  of  two  factors:  the  source  of  the  agency’s  control  (internal 
vs.  external)  and  the  degree  of  control  that  source  has  over  the  agency’s  actions  (high  vs.  low).  See 
Table  20,  opposite. 


Craig  O’Blenis  and  Warren  Nethercote,  pp.  37-38. 

Melvin  J.  Dubnick  and  Jonathan  B.  Justice,  p.  2. 

Barbara  S.  Romzek  and  Melvin  J.  Dubnick,  “Accountability  in  the  Public  Sector:  Lessons  from  the  Challenger 
Tragedy”,  Public  Administration  Review,  47:3  May/June  1987,  p.  229. 
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Table  20:  Accountability  Systems 


168 

Source  of  Control 
Internal  External 

Degree  of  High 
Control 

Low 


Bureaucratic 

Legal 

Professional 

Political 

The  nature  of  DRDC’s  management  hierarchy  and  relationships  places  it  within  the  bureaucratic 
accountability  system.  Under  this  accountability  system,  the  public  servant  receives  priorities  from 
those  at  the  top  of  the  hierarchy,  there  is  a  formal  relationship  between  superior  and  subordinate,  and 
there  are  rules  and  regulations  governing  expectations  and  behaviours. Our  research  and 
subsequent  survey  questions  will  utilize  the  bureaucratic  system  model  of  accountability  as  a 
framework. 

Aucoin  and  Heintzman  identify  three  purposes  of  accountability  in  a  bureaucratic  system:  control, 
assurance  and  continuous  learning.'^®  Accountability  is  a  mechanism  for  exercising  control  over  the 
decision-making  of  subordinates  in  a  system  of  devolved  authority.  Managing  for  and  reporting 
outcomes  and  results  helps  ensure  accountability,  as  these  increase  transparency  and  disclosure 
without  resorting  to  excessive  micro-management. 

In  the  parlance  of  Agency  Theory, accountability  also  frinctions  to  “provide  assurance  to 
principals  that  their  agents  are  fulfilling  their  responsibilities  as  intended.”'’^  To  provide  assurance, 
accountability  regimes  must  specify  “goals  and  objectives,  service  entitlements  and  standards,  best 
management  practices  and  performance  targets”’’"^  and  include  systematic  auditing,  inspection  and 
review.  A  key  principal  of  an  effective  accountability  structure  is  that  while  “individuals,  both  when 
acting  alone  or  in  collaboration,  must  be  held  accountable  for  their  individual  performance”’’^,  there 
is  also  an  acknowledgement  of  “the  constraints  that  affect  the  realization  of  outputs  and  outcomes 
and  over  which  public  servants  have  little  or  no  control.””®  This  is  particularly  important  when 
determining  responsibility  for  events  and  outcomes. 

To  lessen  the  vulnerability  of  each  person  in  the  superior-subordinate  relationship,  it  is  essential  to 
have  objective  measures  of  performance,  so  that  the  subordinate  is  aware  of  the  standard  to  which 
his/her  performance  will  be  held  accountable,  and  the  manager  has  evidence  of  the  subordinate 
having  met  (or  not)  his/her  responsibilities  for  performance. 


Barbara  S.  Romzek  and  Melvin  J.  Dubnick,  p.  229. 

Barbara  S.  Romzek  and  Melvin  J.  Dubnick,  p.  228. 

Peter  Aucoin  and  Ralph  Heintzman,  pp.  45  and  48. 

*’*  Peter  Aucoin  and  Ralph  Heintzman,  pp.  48  -  49. 

Discussed  previously  in  Policy  and  Programs  Element,  page  46  of  this  report. 

Peter  Aucoin  and  Ralph  Heintzman,  pp.  48  -  49. 

Peter  Aucoin  and  Ralph  Heintzman,  pp.  48-  49. 

Peter  Aucoin  and  Ralph  Heintzman,  pp.  48  and  50. 

Peter  Aucoin  and  Ralph  Heintzman,  pp.  48  and  54. 

Elliot  Jaques,  “Chapter  Four:  Managerial  Accountability,  Authority,  and  Dependence”  in  A  General  Theory  of 
Bureaucracy.  New  York:  Halsted  Press,  1976,  p.  79. 


DRDC  Atlantic  CR  2009-135 


89 


Demonstration  of  performance  against  expectations  implies  a  regime  of  reporting  and  review:  the 
subordinate  reports  to  the  superior  and  the  superior  reviews  performance.  In  other  words, 
“accountability  addresses  a  manager’s  responsibility,  first  to  require  a  subordinate  to  report,  and 
then  to  review  the  subordinate’s  performance.’”’®  But  it  is  not  enough  to  just  review  the  performance 
of  subordinates.  To  have  a  true  accountability  regime,  the  superior  must  utilize  the  information 
gathered  from  the  review  to  benefit  the  organization,  otherwise,  no  real  purpose  is  served  by  the 
accountability  structure.  In  the  words  of  Aucoin  and  Heintzman,  “those  responsible  for  managing 
the  managers. . .  [must]  ensure  that  accountability  for  individual  management  performance 
counts”.'™ 

In  addition  to  focusing  on  control  and  assurance,  the  goal  of  an  accountability  regime  is  also  to 
support  continuous  improvement  in  policy,  the  organization  or  its  management.  Results  of 
individual  performance  assessments  (both  positive  and  negative)  should  be  utilized  as  learning 
opportunities.'®' 

[A]  ccountability  can,  and  should,  be  a  major  force  for  improving  performance. 

Elliot  Jaques  describes  the  accountability  regime  as  a  three  level  managerial  linkage,  depicted  in 
Figure  40,  where  “the  manager  is  accountable  to  the  manager-once-removed  for  his  own  work  and 
for  the  work  of  his  subordinate,  and  he  manages  his  subordinate  within  the  terms  of  reference  set  by 
the  manager-once-removed.” The  manager-once-removed  is,  in  turn,  accountable  to  the 
Governing  Body  for  his  own  work  and  for  the  work  of  his  subordinate,  the  manager.  The  manager 
may  delegate  responsibilities  to  subordinates  (in  fact,  ordinarily  must  do  so)  but  cannot  escape 
accountability  to  his  own  manager  (the  manager-once-removed)  for  those  delegated  responsibilities 
and  authorities.  The  manager  must  have  requisite  delegated  authority  from  the  manager-once- 
removed  to  exploit  the  resources  assigned,  whether  human  or  financial.  It  has  been  argued  that  “in 
fact,  accountability  should  be  viewed  as  that  which  is  exchanged  for  the  devolution  of  authority.  It 
can  be  a  positive  opportunity  to  demonstrate  performance  results  against  the  expectations  of  those 
who  have  faith  and  trust  in  an  individual,  an  element  of  the  organization,  or  the  organization 
itself”'®® 


178  Peter  Aucoin,  private  communication,  July  7,  2008. 

Peter  Aucoin  and  Ralph  Heintzman,  p.  5 1 . 

'®'’  Peter  Aucoin  and  Ralph  Heintzman,  p.  52. 

'®'  Peter  Aucoin  and  Mark  D.  Jarvis,  Modernizing  Government  Accountability:  A  Framework  for  Reform,  Canada  School 
of  Public  Service,  2005,  p.  9. 

'®’  Peter  Aucoin  and  Ralph  Heintzman,  p.  47. 

'®®  Elliot  Jaques,  p.  65. 

'®"' Elliot  Jaques,  p.  65. 

'®®  Department  of  National  Defence,  Strengthening  Accountability  and  Comptrollership  in  National  Defence,  June  2004,  p. 
C8/22,  available  from:  http://www.admfincs.forces.gc.ca/sacnd-rrfcdn-eng.asp. 
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Figure  40:  Jacques'  Three-level  Managerial  Linkage‘^^ 

This  manager-subordinate  relationship  has  the  following  features: 

The  subordinate  has  eontracted  to  carry  out  . .  .delegated  tasks  and  must  do  so; 

The  subordinate  is  . . .  vulnerable  to  the  manager’s  assessment  of  his  performance; 

A  manager  is  . . .  vulnerable  to  the  quality  of  his  subordinate’s  performance. 

Frameworks  and  systems  describe  the  environments  in  which  accountability  works,  or  is  required, 
but  do  not  define  accountability  per  se.  Definitions  of  accountability  typically  include  reference  to 
“willingness  to  accept  responsibility  or  to  account  for  one's  actions.”'**  However,  it  can  be  argued 
that  this  definition  of  accountability  neglects  much  of  the  word’s  more  subtle  yet  important  aspects: 
liability,  answerability,  responsiveness,  obligation,  obedience,  fidelity  and  amenability.  These 
aspects  are  not  synonymous  with  accountability,  rather  they  form  a  part  of  the  accountability 
concept,  and  particular  aspects  may  figure  more  prominently  depending  on  the  context  in  which  the 
term  accountability  is  utilized. 

Within  the  bureaucratic  system,  the  definition  of  accountability  provided  by  Aucoin,  Smith,  and 
Dinsdale  eloquently  incorporates  the  aspects  of  accountability  that  are  necessary  for  good 
governance: 

Accountability  [is]  to  justify,  explain  or  defend  one’s  actions  (or  those  of  one’s 
subordinates)  based  on  powers  and  responsibilities  bestowed  by  a  superior 
authority.  The  account  may  encompass  a  statement  of  any  necessary  corrective 


186  Elliot  Jaques,  p.  63. 

187  Elliot  Jaques,  p.  66. 

188  Merriam-Webster  On-line  Dictionary,  available  from:  httri://www.merriam- webster.com/dictionarv/accountabilitv. 

189  Melvin  J.  Dubnick  and  Jonathan  B.  Justice,  pp.  5-7. 
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action  to  be  taken.  The  superior  authority  has  the  obligation  to  hold  to  account 
all  those  on  whom  it  has  bestowed  powers  and  responsibilities.'’” 

Responsibility  is  a  key  feature  of  this  definition  of  accountability  and  it  is  important  to  note  again 
that  responsibility  is  not  synonymous  with  accountability.  Rather,  consider  that  responsibilities  are 
delegated  to  subordinates  along  with  the  authority  necessary  to  reasonably  perform  these 
delegations;  in  exchange  the  subordinate  must  account  to  his/her  superior  for  his/  her  performance 
of  these  responsibilities.  The  Somalia  Inquiry  attempted,  without  much  success,  to  clarify  the 
difference  as  follows: 

One  who  is  authorized  to  act  or  exercise  (sic)  authority  is  ‘responsible.’ 

Responsible  officials  are  held  to  account.  An  individual  who  exercises  powers 
while  acting  in  the  discharge  of  official  functions  is  responsible  for  the  proper 
powers  or  duties  assigned.'” 

This  murky  explanation  offered  in  Dishonoured  Legacy  is  unfortunate  given  the  importance  of  the 
inquiry  to  the  organizational  development  of  the  Department  of  National  Defence  and  the  Canadian 
Forces. 

In  the  document  Organization  and  Accountability,  the  Department  of  National  Defence  solidly  links 
the  triad  of  responsibility,  authority  and  accountability,  while  introducing  the  concept  that  delegated 
authorities  and  responsibilities  bring  with  them  the  implicit  obligation  to  act: 

Having  a  [delegated]  responsibility  involves  having  the  [delegated]  authority  and 
the  obligation  to  act,  including  the  authority  to  direct  or  authorize  others  to  act. 

It  also  means  being  accountable  for  how  those  responsibilities  have  been  carried 
out  in  light  of  agreed  or  set  expectations,  particular  duties,  or  obligations.  In  a 
public  sector  organization  ...  each  individual  is  obliged  to  account  fully  and 
promptly  to  those  who,  in  the  hierarchy,  conferred  the  responsibilities,  for  the 
way  they  have  been  carried  out  and  for  how  the  relevant  authorities  have  been 
used.'’^ 

The  present  authors  have  added  “[delegated]”  to  the  quotation  to  stress  that  responsibility  and 
authority  are  typically  bestowed  by  a  superior,  who  will  hold  the  actions  of  the  delegate  to  account, 
as  per  Aucoin,  Smith  and  Dinsdale. 

The  accountability  element  of  the  proposed  DRDC  MAF  will  use  Jaques’  accountability  regime, 
along  with  Aucoin  and  Heinzman’s  tri-partite  explanation  of  accountability’s  purposes,  as  its 
foundation. 


Peter  Aucoin,  Jennifer  Smith,  and  Geoff  Dinsdale,  Responsible  Government:  Clarifying  essentials,  dispelling  myths  and 
exploring  change,  Ottawa:  Canadian  Centre  for  Management  Development,  2004,  p.  88. 

'’'  Government  of  Canada,  Report  of  the  Commission  of  Inquiry  into  the  Deployment  of  Canadian  Forces  to  Somalia, 
Dishonoured  Legacy:  The  Lessons  of  the  Somalia  Affair,  Ottawa:  Public  Works  and  Government  Services  Canada,  1997, 
p.  393. 

'’^Department  of  National  Defence,  Organization  and  Accountability:  Guidance  for  Members  of  the  Canadian  Forces  and 
Employees  of  National  Defence,  second  ed.,  September  1999,  p.  2,  available  from: 
http  ://www.  forces,  gc  .ca/  admpol/Organization-e.html . 


92 


DRDC  Atlantic  CR  2009-135 


Accountability:  Expectations 

The  Expectations  Tier  of  the  DRDC  MAE  was  developed  in  Cycle  1,  based  on  the  TB  MAE  and 
consultation  with  key  DRDC  personnel.  Figure  41  illustrates  the  Accountability  component  of  the 
DRDC  MAE  Expectations  tier. 


-1 

II 

Accountability 

Accountabilities  for  results  are  clearly 
assigned  and  are  consistent  with 
resources,,  and  delegations  are 
consistent  with  c^abilities. 

Figure  41:  Accountability,  Expectations  Tier 


Accountability:  Indicators 

The  TB  Accountability  Indicators  Tier  has  three  elements;  As  per  the  previous  section  on  Risk 
Management,  the  DRDC  Indicators  for  Accountability  are  based  on  those  of  the  TB  MAE.  The 
Indicators  are  tailored  to  reflect  the  DRDC  environment  and  level  of  management  utilizing  the 
MAE. 

Clear  accountabilities  and  responsibilities  for  due  process  and  results.  This  element  clear  and 
applicable  and  has  been  transferred  directly  to  the  indicator  “Clear  accountabilities  and 
responsibilities  for  due  process  and  results”.  Due  process  can  be  thought  of  as  referring  to  duties 
which  are  included  in  a  subordinate’s  job  description  (including  authorities  delegated  by  FAA), 
while  responsibilities  for  results  are  tied  to  tasks  assigned  to  an  individual  by  a  superior.  193  The 
subordinate  is,  in  fact,  accountable  for  both  duties  and  tasks,  but  the  chain  of  accountability  may 
differ.  Additionally,  the  due  process  and  results  relationship  suggests  that  public  servants  are 
accountable  not  only  for  the  outcome  of  an  initiative,  but  also  for  how  the  outcome  was  achieved,  or 
simply  put  ‘the  ends  do  not  justify  the  means’. 

Delegations  appropriate  to  capabilities.  This  element  is  applicable,  but  for  the  sake  of  clarity  has 
been  amended  to  the  indicator  '"Delegations  consistent  with  capabilities.” 

Cascading  commitments  in  Performance  Management  Agreements.  The  term  ‘Performance 
Management  Agreement  (PMA)  is  not  widely  used  outside  of  executive  (Eevel  1)  management,  so 
would  be  of  little  value  to  managers  at  DRDC.  Instead,  managers  would  have  performance 
objectives  or  goals,  and  the  assignment  of  tasks  to  subordinates  should  support  those  goals.  This 
element  is  captured  in  the  indicator  "Subordinates  ’  goals  reflect  those  assigned  to  their 
supervisors.” 


Peter  Aucoin,  private  communication,  July  7,  2008. 
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Figure  42,  below,  depicts  the  Indicators  for  Accountability  in  the  DRDC  MAF. 


Accountability 

Clear  accountabilities  and  responsibilities  firr 
due  process  and  results 

Delegations  consistent  with  curabilities  - ^ 

Subordinates'  goals  reflect  those  assigned  to 
their  supervisors 

Figure  42:  Accountability,  Indicators  Tier 


Accountability:  Measures 

The  Measures  Tier  of  the  TB  MAF  includes  four  measures  for  Accountability: 

Clarity  of  accountabilities; 

Delegations  regularly  reviewed; 

Executive  committee  oversight  of  performance  management  and  regular  review  of  performance; 
and. 

Alignment  of  individual  with  corporate  commitments. 

Although  accountability  tends  to  focus  on  individual  performance,  the  questions  in  the  survey 
instrument  must  also  address  organizational  performance.  The  Stewardship  element  addresses 
delegated  authority,  and  as  a  result,  it  will  not  be  covered  in  Accountability.  There  is  no  one-to-one 
correspondence  between  the  Accountability  measures  of  the  TB  MAF  and  those  of  the  DRDC  MAF. 

Figure  43,  below,  depicts  the  Measures  for  Accountability  in  the  DRDC  MAF. 


TBS,  Management  Accountability  Framework. 
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II 

Accountability 

Management  goals  are  identifiable  and 
measuieable 

Duties  and  tasks  are  clear  to  subordinates 

Managers  and  staff  have  ^rpropiiate  training 
and  authority  to  undertake  their  delegations 

A  firrmal  review  process  tracks  progress, 
delegations  and  perfimnance 

Achievement  of  subordinates'  goals  supports 
management  goals  and  the  organizational 
vision 

Figure  43:  Accountability,  Measures  Tier 


Table  21  presents  examples  of  information  which  can  be  used  as  evidence  of  each  measure. 


Table  21:  Evidence  and  Instruments  for  Accountability 


Measure:  Management  goals  are  identifiable  and  measureable 

Evidence 

Source/  Instruments 

•  Goals  are  set  by  or  agreed  on  with  superior  managers 

•  Goals  are  expressed  to  identify  measurable  products. 

•  Desired  outcomes  are  documented 

•  Identified  outcomes  correspond  to  the  assigned  tasks 

•  Performance  Management 
Program  (PMP)  Annual  or 
Triennial  Scorecard 

•  PMP  for  Level  2  Managers 

•  Management/annual 
objectives  for  Levels  3  and  4 
(assessment  of  local  practice) 

•  Managers’  survey 

•  Subordinate  PLP’s 

Measure:  Duties  and  tasks  are  clear  to  subordinates 

Evidence 

Source/  Instruments 

•  Assigned  tasks  are  communicated  effectively. 

•  The  duties  associated  with  a  position  are  documented  and 
accessible. 

•  Subordinates  are  able  to  seek  clarification  of  duties  and/or 
tasks  from  superiors  and/or  Corporate  Services. 

•  Current  statements  of  duties 

•  Measureable  objectives 

•  Timely  performance 
assessment,  linked  to 
objectives 
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Measure:  Managers  and  staff  have  appropriate  training  and  authority  to  undertake  their 
delegations 

Evidence 

Source/  Instruments 

•  Managers  have  received  the  training  necessary  to  execute 
the  delegations  normally  associated  with  their  position. 

•  Personal  Learning  Plans  inform  training  and  goal 
assignment. 

•  Records  of  managers’ 
certifications 

•  Records  of  mandatory 
management  training 

Measure:  A  formal  review  process  tracks  progress,  delegations  and  performance 

Evidence 

Source/  Instruments 

•  All  employees  undergo  routine  Performance  Appraisals 

•  Milestones  are  identified  to  monitor  progress  towards  goals 

•  Performance  appraisal 
statistics 

Measure:  Achievement  of  subordinates'  goals  supports  management  goals  and  the  organizational 
vision 

Evidence 

Source/  Instruments 

•  goals,  if  achieved,  will  support  the  achievement  of  a 
superior’s  goals. 

•  goals  include  elements  tied  to  milestones  identified  in 

CPME  (the  DRDC  program  formulation  tool)  for  which  the 
manager  is  responsible. 

•  Managers’  survey 
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6  Survey  Instrument  Design 


The  MAT  is  intended  for  use  as  an  organizational  development  and  process  improvement  tool. 
Annually,  the  Agency  will  gather  information  from  a  variety  of  sources  to  assess  its  performance 
against  the  1 0  elements  of  the  MAP  to  aid  in  reporting  to  the  ADM.  One  of  the  information  sources 
utilized  will  be  a  Manager’s  survey.  The  survey  approach  is  intended  to  gather  information  on  how 
DRDC  managers  at  relevant  operational  levels  function  in  the  framework  of  the  ten  MAP  elements. 

6.1  Cycle  2  Survey  Design 

The  areas  of  the  MAP  reviewed  through  a  survey  approach  in  this  cycle  were  Risk  Management, 
Accountability,  Policy  &  Programs,  and  Service;  Stewardship  was  also  reassessed.  These  elements 
were  assessed  through  two  separate  surveys.  The  first  -Pilot  Survey  2-  encompassed  Risk 
Management  and  Accountability.  The  second  -Pilot  Survey  3-  assessed  Policy  &  Programs  and 
Service,  and  re-assessed  Stewardship.  The  People  element  was  not  addressed  in  these  surveys,  as 
relevant  information  can  be  accessed  through  the  Public  Service  Employee  Survey,  2005  and 
2008. 

Por  our  purposes.  Levels  2,  3  and  4  were  the  target  population.  ‘Levels’  are  as  defined  or  implied,  in 
the  DND  and  DRDC  delegation  matrices. 

The  positions  which  comprise  these  Levels  are  listed  in  Table  22. 

Table  22:  DRDC  Management  Levels 


Organizational 

Level 

Incumbents 

Level  1 

CEO  DRDC 

Level  2 

Directors  General,  Chief  of  Staff/ADM(S&T) 

Level  3 

Deputy  Directors  General,  Chief  Scientists,  Directors,  Scientific 
Section  Heads,  Managers 

Level  4 

Punctional  Authorities,  Supervisors 

CPSA,  Public  Service  Employee  Survey  2005;  TBS,  2008  Public  Service  Employee  Survey  Results. 
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Pilot  Survey  2  Design 


Pilot  Survey  2  questions  were  developed  based  on  DRDC  MAP  Expectations,  Indicators,  and 
Measures  Tiers  for  Risk  Management  and  Accountability,  as  well  as  input  from  pre-existing, 
publicly  available  survey  instruments. 

Survey  questions  were  developed  to  address  each  of  the  measures  in  Risk  Management  and 
Accountability.  The  majority  of  questions  were  closed  so  that  respondents  could  choose  only  from 
pre-determined  alternatives.  Where  appropriate,  respondents  were  given  the  option  to  write  in 
alternate  responses  (open  questions).  The  survey  questions  were  then  checked  for  redundancy, 
clarity,  and  brevity,  and  adjusted  where  appropriate.  Subject  matter  experts  reviewed  a  draft  survey; 
we  utilized  their  input  to  adjust  the  survey  content  and  presentation.  At  this  point,  the  survey  was 
sent  out  for  translation  into  French. 

The  final  set  of  questions  were  sorted  into  a  logic  tree  to  ensure  each  respondent  was  only  asked 
questions  applicable  to  his/her  job  level  and  duties.  The  bulk  of  questions  are  aimed  at  Level  3 
managers,  and  to  a  lesser  extent.  Level  4  managers.  Level  2  managers  are  asked  comparatively  few 
questions.  This  is  by  design,  in  part  because  the  DRDC  MAP  is  geared  towards  lower  level 
managers,  and  also  because  Level  2  managers  are  few  in  numbers,  challenging  the  need  for 
anonymity.  Thus  we  did  not  ask  more  sensitive  performance  related  questions  to  Level  2  managers. 

A  paper  based  survey  was  tested  by  a  small  subset  of  the  target  population.  Information  from  our 
test  group  was  used  to  refine  the  survey  for  the  online  version.  We  tested  the  on-line  version  of  the 
survey,  and  made  minor  adjustments  to  the  logic  tree.  The  on-line  survey  was  then  re -tested  by  the 
subset.  Finally,  the  survey  was  reviewed  for  compliance  to  internal  research  standards  by  the  DND 
Social  Science  Research  Review  Board  through  DGMPRA. 

The  survey  was  distributed  to  all  141  of  the  Level  2,  Level  3,  and  Level  4  functional  authorities  who 
were  eligible  to  attend  the  DRDC  Managers  Workshop,  as  of  February  1,  2009.  The  questionnaire 
was  implemented  bilingually  on  the  DRDC  Atlantic  ORACLE®  server.  Survey  participants  were 
provided  with  a  link  that  could  be  accessed  from  anywhere  within  the  DRENet.  Pilot  Survey  2  was 
launched  through  DRDC  Atlantic  Oracle®  server  on  Wednesday  February  4,  2009,  and  closed  on 
Monday  February  16,  2009. 

Survey  2  Limitations 

There  are  a  number  of  limiting  factors  in  the  design  and  implementation  of  this  survey.  Paramount 
among  these  is  that  DND  has  not  yet  released  an  approved  Corporate  Risk  Profile  (CRP).  Existence 
of  a  CRP  and  distribution  throughout  the  organization  is  a  cornerstone  of  the  MAP  risk  management 
expectation;  without  this  strategic  document,  DRDC  cannot  possibly  be  expected  to  align  its  RM 


***  CRS-DND,  “Annex  B-  Diagnostic  Tool”  in  Baseline  Study:  Integrated  Risk  management  within  the  DND/CF',  Audit 
Office  of  New  South  Wales,  “Appendix:  Questionnaire  Risk  Managemenf  ’  m  Performance  Audit  Report:  Managing  Risk 
in  the  NSW  Public  Sector,  Barbara  D.  Adams,  Sonya  Waldherr,  and  Kenneth  Lee,  “Annex  E”  in  Interoperable  Risk 
Management  in  a  Joint  Interagency  Multinational  Environment,  DRDC  CR2007-068,  August  2007;  Canada  Public 
Service  Agency,  Public  Service  Employee  Survey  2005,  available  from: 

http://www.collectionscanada.gc.ca/webarchives/200711 15073357/httD://www.Dsagencv-agencefp.gc.ca/survev- 

sondage/2005/index  e.asp 
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strategy  with  that  of  DND.  The  DND  CRP  is  in  the  final  stages  of  development  and  is  expected  to 
be  given  Ministerial  approval  by  the  end  of  2009. 

DRDC  appears  to  lack  a  comprehensive  RM  strategy;  there  is  neither  a  process  in  place  to  identify 
stakeholders,  nor  a  communication  plan  to  communicate  with  stakeholders.  Stakeholder 
identification  appears  to  be  based  on  corporate  memory  of  past  consultations;  this  knowledge  should 
be  captured  in  a  data  system,  and  an  approach,  such  as  an  environmental  scan,  should  be  developed 
to  identify  stakeholders  who  may  not  have  been  considered  in  the  past. 

Pilot  Survey  3  Design 

Pilot  Survey  3  questions  were  developed  based  on  DRDC  MAP  Expectations,  Indicators,  and 
Measures  tiers  for  Policy  and  Programs  and  Service,  as  well  as  input  from  pre-existing,  publicly 
available  survey  instruments  and  policy  guides.  Questions  about  the  Stewardship  element  are 
based  on  the  questions  and  results  of  Pilot  Survey  1 . 

Survey  questions  were  developed  to  address  each  of  the  measures  in  Policy  and  Programs,  and 
Service;  for  the  Stewardship  element,  questions  focused  on  areas  of  concern  from  the  Cycle  1  survey 
analysis.  The  majority  of  questions  were  closed  so  that  respondents  could  choose  only  from  pre¬ 
determined  alternatives.  Where  appropriate,  respondents  were  given  the  option  to  write  alternate 
responses  (open  questions).  As  with  Survey  2,  the  survey  questions  were  then  checked  for 
redundancy,  clarity,  and  brevity,  and  adjusted  where  appropriate.  Subject  matter  experts  reviewed 
the  draft  survey;  we  utilized  their  input  to  adjust  the  survey  content  and  presentation,  and  then  had 
the  final  survey  translated  into  French. 

The  final  set  of  questions  were  sorted  into  a  logic  tree  to  ensure  each  respondent  was  only  asked 
questions  applicable  to  his/her  job  level  and  duties.  Depending  on  the  respondent’s  responsibilities, 
questions  ranged  from  a  minimum  of  16  to  a  maximum  of  59  questions  to  complete,  taking  from  15 
to  40  minutes. 

A  paper  based  survey  was  tested  by  a  small  subset  of  the  target  population.  Information  from  our 
test  group  was  used  to  refine  the  survey  for  the  online  version.  We  tested  the  online  version  of  the 
survey,  and  made  minor  adjustments  to  the  logic  tree.  The  online  survey  was  then  re-tested  by  the 
subset.  Finally,  the  survey  was  reviewed  for  compliance  to  internal  research  standards  by  the  Social 
Science  Research  Review  Board  through  DGMPRA. 

The  survey  was  distributed  to  all  147  of  the  Fevel  2,  Fevel  3,  and  Fevel  4  functional  authorities  who 
were  eligible  to  attend  the  DRDC  Managers  Workshop,  as  of  May  15,  2009.  The  questionnaire  was 
implemented  bilingually  on  the  DRDC  Atlantic  ORACFE®  server.  Survey  participants  were 
provided  with  a  link  that  could  be  accessed  from  anywhere  within  the  DRENet.  Pilot  Survey  3  was 
launched  through  DRDC  Atlantic  Oracle®  server  on  Friday  May  29,  2009,  and  closed  on  Friday 
June  12,  2009. 


Office  of  the  Deputy  Prime  Minister,  Assessing  Strategic  Partnerships:  The  Partnership  Assessment  Tool,  [AKA 
Nuffield  Partnership  Assessment  Tool],  United  Kingdom,  2003,  available  from: 

http://www.communities.gov.uk/publications/localgovernment/assessingstrategicDartnership:  TBS,  Succession  Planning 
and  Management  Guide. 
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Assessing  Measures 


A  number  of  questions  have  response  sets  utilizing  a  Likert  scale.  The  most  common  Likert  scale  in 
our  survey  is  that  of  frequency,  ranging  from  ‘Always’  to  ‘Never’.  For  purposes  of  analysis, 
responses  are  sometimes  grouped  to  reflect  consistency,  so  that  responses  of  Always  and  Often  are 
considered  ‘Consistent’,  while  responses  of  Sometimes  and  Rarely  are  considered  ‘Inconsistent’;  a 
response  of  Never  remains  classified  as  Never. 

For  Likert  Scales  utilizing  level  of  agreement,  where  responses  of  ‘Strongly  Agree’  and  ‘Agree’ 
together  total  80%  or  more  of  the  valid  responses  to  a  single  question,  the  overall  response  to  the 
question  is  considered  ‘Positive’. 

To  test  relationships  between  variables,  we  have  chosen  to  measure  statistical  significance  against  a 
minimum  threshold  of  95%  confidence,  where  the  critical  value  a  =  0.05,  so  that  p  (the  significance 
of  the  relationship  between  the  variables)  must  be  <  0.05;  SPSS  reports  the  value  of  j?  as  a  statistic, 
‘Approximate  Significance’.  For  relationships  between  variables  to  be  considered  statistically 
significant,  the  ‘Approximate  Significance’  must  be  between  0.000  (meaning  there  is  less  than 
0.001%  chance  the  result  is  due  to  sampling  error,  corresponding  to  a  relationship  at  the  99.9% 
confidence  level)  and  0.050  (meaning  there  is  a  5%  chance  the  result  is  due  to  sampling  error, 
corresponding  to  a  relationship  at  the  95%  confidence  level). 

In  most  cases,  Kendall’s  tau-b  or  tau-c  tests  are  utilized  to  analyse  relationships  between  two  given 
ordinal  variables.  Cramer’s  V  is  also  reported,  to  give  additional  information  on  the  magnitude  and 
significance  of  any  relationship.  Kendall’s  tau-b  or  tau-c  values  have  been  assessed  according  to  the 
following  scale  in  Table  23. 


Table  23:  Ordinal  Relationship  Assessment  Scale 


Value  (absolute) 

Descriptor 

0.0-0.099 

No  relationship 

0.1-0.199 

Weak  relationship 

0.2-0.299 

Moderate  Relationship 

0.3-0.399 

Moderate-  Strong  Relationship 

0.4-0.499 

Strong  Relationship 

0.5  and  higher 

Very  Strong 

100 
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7  Survey  Analysis 


For  the  convenience  of  the  reader,  statistical  outputs  presented  have  been  numbered  consecutively 
throughout  Section  7,  using  the  same  convention  as  numbering  of  equations. 

7.1  Pilot  Survey  2  Results 

Of  the  141  survey  invitations  solicited,  there  were  89  responses,  of  which  75  were  valid;  this  is  a 
valid  response  rate  of  53%. 

Response  rates  by  management  level  were: 

L2:  5  valid  responses  of  1 1  possible  (45%  return) 

L3:  63  valid  responses  of  100  possible  (63%  return) 

L4:  7  valid  responses  of  30  possible  (23%  return) 

Response  rates  for  Level  4  Functional  Authorities  are  too  small  to  be  considered  generalizable,  but 
their  data  will  be  used  as  a  basis  of  discussion  points  for  future  analysis.  Level  2  Managers’  data  can 
only  be  reported  in  non-identifying  parameters. 

The  logic  tree  within  the  questionnaire  resulted  in  a  varying  number  of  questions  for  respondents: 
the  minimum  was  7  and  the  maximum  was  49  for  completed  surveys. 

Results  are  presented  according  to  management  level  of  the  respondents. 

Level  2  Managers 

Four  of  five  Level  2  respondents  (80%)  agree  that  their  centre’s  organizational  chart  accurately 
reflects  the  reporting  structure  (Q3);  the  remaining  respondent  cited  some  challenges  due  to  the 
relationship  between  the  DRDC  organization  and  DND. 

Overall,  Level  2  Managers  appear  to  have  confidence  that  their  subordinates  understand  their  roles, 
responsibilities,  and  authority. 


Q4  Do  you  have  confidence  subordinates  understand  their  roies? 

Frequency 

Percent 

Valid  Percent 

Always 

1 

20.0 

20.0 

Valid  Often 

4 

80.0 

80.0 

Total 

5 

100.0 

100.0 
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Q5  Do  you  have  confidence  subordinates  understand  their 
responsibiiities? 

Frequency 

Percent 

Valid  Percent 

Always 

3 

60.0 

60.0 

Valid  Often 

2 

40.0 

40.0 

Total 

5 

100.0 

100.0 

(2) 


Q6  Do  you  have  confidence  subordinates  understand  their 

authority? 

Frequency 

Percent 

Valid  Percent 

Valid  Often 

5 

100.0 

100.0 

(3) 


All  L2  managers  (100%)  report  setting  performance  goals  for  their  subordinates  (Q7),  but  are 
divided  as  to  applying  measurable  standards  to  those  goals  consistently. 


Q9  Do  subordinates'  performance  objectives  inciude  measurabie 

standards? 

Frequency 

Percent 

Valid  Percent 

Valid  Often 

3 

60 

60 

Rarely 

2 

40 

40 

Total 

5 

100 

100 

Level  2  managers  report  basing  subordinates’  performance  objectives  on  their  own  performance 
objectives  (100%)  and  their  superior’s  expectations  of  performance  (100%);  employees’  Personal 
Learning  Plans  (80%);  Corporate  Initiatives  (80%);  past  year’s  performance  (40%);  and  employees’ 
statement  of  duties  (20%).  None  of  the  Level  2  respondents  base  performance  objectives  with 
reference  to  CPME  milestones. 


Q8  How  do  you  set  performance  objectives  for  subordinates? 

Yes  (%) 

No  (%) 

based  on  my  superior's  expectations 

100 

0 

based  on  subordinates'  PLPs 

80 

20 

based  on  corporate  initiatives 

80 

20 

based  on  other 

80 

20 

based  on  past  years'  performance 

40 

60 

based  on  statement  of  duties 

20 

80 

based  on  CPME  milestones 

0 

100 

‘Other’  basis  for  setting  performance  objectives  can  be  broadly  stated  as  “alignment  with  identified 
goals  and  priorities  at  a  strategic  level,”  specifically,  these  include: 
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Performance  Management  Agreements; 

Functional  planning  priorities; 

Convening  Letter; 

Level  1  priorities;  and, 

Centre-level  goals  and  initiatives. 

All  Level  2  managers  (100%)  report  that  regardless  of  how  performance  objectives  are  set, 
subordinates’  objectives  consistently  support  the  performance  objectives  of  their  superiors.  This  is 
consistent  with  the  roll  up  nature  of  the  Accountability  element. 


Q10  Do  subordinates'  performance  objectives  support  your  own 
performance  objectives? 

Frequency 

Valid  Percent 

Always 

3 

60 

Often 

2 

40 

Total 

5 

100 

(6) 


Level  3  Managers 

Level  3  managers  were  the  primary  respondents  in  the  survey.  They  were  the  largest  part  of  the 
population,  with  63  valid  responses  of  141  eligible  participants,  and  were  the  target  of  the  greatest 
number  of  questions. 

Accountability:  Performance  measures  and  management 

Most  Level  3  respondents  report  that  their  performance  is  reviewed  through  a  combination  of 
methods,  most  commonly  against  general  expectations,  salary  administration  systems  and  agreed 
performance  objectives. 


Q  11  How  does  your  superior  review  your  performance? 

Yes  (%) 

No  (%) 

against  salary  admin  system  or 
performance  management  framework 

69.8 

30.2 

against  general  expectations 

60.3 

39.7 

against  agreed  performance  objectives 

50.8 

49.2 

against  statement  of  duties 

39.7 

60.3 

against  my  PLP 

25.4 

74.6 

on  an  ad  hoc  basis 

14.3 

85.7 

(7) 
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Although  four  respondents  selected  ‘other’  as  the  method  by  which  their  performance  is  reviewed, 
the  comments  actually  indicate  that  the  methods  fit  into  established  categories,  and  so  were  included 
in  the  appropriate  categories  above. 

Forty  percent  (40%)  of  respondents  report  that  their  superior  consistently  (Always  or  Often)  sets 
performance  objectives  for  them,  while  54%  inconsistently  (Sometimes  or  Rarely)  have  objectives 
set,  and  the  remaining  6%  never  have  performance  objectives  set  by  their  superior. 


Q12  Does  your  superior  set  performance  objectives  for  you? 

Frequency 

Percent 

Valid  Percent 

Valid  Always 

10 

15.9 

15.9 

Often 

15 

23.8 

23.8 

Sometimes 

20 

31.7 

31.7 

Rarely 

14 

22.2 

22.2 

Never 

4 

6.3 

6.3 

Total 

63 

100.0 

100.0 

For  those  who  do  have  performance  objectives  set  by  superiors,  more  than  half  (64%)  do  not 
consistently  include  any  measurable  standards. 


Q13  Do  your  performance  objectives  inciude  measurabie  standards? 

Frequency 

Percent 

Valid  Percent 

Valid  Always 

5 

7.9 

8.5 

Often 

16 

25.4 

27.1 

Sometimes 

23 

36.5 

39.0 

Rarely 

14 

22.2 

23.7 

Never 

1 

1.6 

1.7 

Total 

59 

93.7 

100.0 

Missing  99 

4 

6.3 

Total 

63 

100.0 
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Level  3  managers  who  inconsistently  have  measurable  standards  as  a  part  of  their  performance 
objectives  appear  more  likely  to  inconsistently  set  measurable  standards  as  a  part  of  their 
subordinates’  performance  objectives,  as  seen  by  comparing  results  of  questions  13  and  35. 


Q35  Do  your  subordinates'  performance  objectives  inciude  measurabie  standards?  *  Q13  Do  your 
performance  objectives  inciude  measurabie  standards? 

Do  your  performance  objectives  include  measurable 
standards? 

Always 

Often 

Sometimes 

Rarely 

Never 

Total 

Do  your 

Always 

1 

3 

0 

0 

0 

4 

subordinates' 

performance 

Often 

3 

7 

13 

6 

0 

29 

objectives  include 

Sometimes 

0 

4 

8 

3 

1 

16 

measurable 

standards? 

Rarely 

0 

0 

1 

5 

0 

6 

Total 

4 

14 

22 

14 

1 

55 

(10) 


A  non-parametric  Kendall’s  tau  -c  test  indicates  there  is  a  moderate-strong  positive  relationship 
between  the  rankings  of  the  two  variables,  and  that  the  relationship  is  statistically  significant  at 
99.9%  confidence. 


Q13  Do  your  performance  objectives  inciude  measurabie  standards?  * 
Q35  Do  your  subordinates'  performance  objectives  inciude  measurabie 

standards? 

Value 

Approx. 

Sig. 

Ordinal  by  Ordinal 

Kendall’s  tau-c 

0.344 

0.000 

Nominal  by  Nominal 

Cramer’s  V 

0.381 

0.020 

Almost  thirteen  percent  (12.7%)  of  Level  3  respondents  are  unaware  of  how  their  own  objectives 
might  impact  their  superior’s  performance  objectives,  while  23.7%  believe  their  objectives  are 
inconsistent  in  supporting  the  objectives  of  their  superior. 


Q14  Do  your  performance  objectives  support  your  superior's  own 
performance  objectives? 

Frequency 

Percent 

Valid 

Percent 

Valid  Always 

15 

23.8 

25.4 

Often 

22 

34.9 

37.3 

Sometimes 

11 

17.5 

18.6 

Rarely 

3 

4.8 

5.1 

Uncertain/  Don't 
Know 

8 

12.7 

13.6 

Total 

59 

93.7 

100.0 
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Level  3  managers  are  divided  in  the  consistency  of  setting  subordinates’  performance  objectives  to 
support  their  own  objectives  (52.7%  consistently  vs.  47.4%  inconsistently). 


Q34  Do  your  subordinates'  performance  objectives 
support  your  own  performance  objectives? 

Frequency 

Valid  Percent 

Always 

7 

12.3 

Often 

23 

40.4 

Valid  Sometimes 

26 

45.6 

Rarely 

1 

1.8 

Total 

57 

100.0 

Level  3  managers  who  consistently  have  performance  objectives  which  support  their  superior’s 
objectives  appear  more  likely  to  consistently  set  subordinates’  performance  objectives  so  that  they 
support  their  own  objectives.  This  is  consistent  with  a  roll-up  approach  of  Program  Activity 
Architecture.  A  non-parametric  Kendall’s  tau-c  test  indicates  there  is  a  strong  positive  relationship 
between  the  rankings  of  the  two  variables,  and  that  the  relationship  is  statistically  significant  at 
99.9%  confidence. 


Q34  Do  your  subordinates’  performance  objectives  support  your  own  performance 
objectives?  *  Q14  Do  your  performance  objectives  support  your  superior's  own 

performance  objectives? 

Do  your  performance  objectives 
support  your  superior's  own 
performance  objectives? 

Always  /  Often 

Sometimes  / 
Rarely 

Always  / 

Do  your  subordinates'  performance  Often 

objectives  support  your  own 

performance  objectives?  Sometimes  / 

Rarely 

64% 

36% 

29% 

71% 

Total 

100% 

100% 

(14) 


Q  14  Do  your  performance  objectives  support  your  superior's  own  performance 
objectives?  *  Q34  Do  your  subordinates'  performance  objectives  support  your  own 

performance  objectives? 

Value 

Approx.  Siq. 

Ordinal  by  Ordinal 

Kendall’s  tau-b 

0.489 

0.000 

Nominal  by  Nominal 

Cramer’s  V 

0.552 

0.000 

(15) 
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Level  3  managers  whose  subordinates  are  given  performance  objectives  which  roll  up  to  support 
their  own  performance  objectives  also  appear  more  likely  to  set  measurable  standards  for  their 
subordinates’  performance.  A  Kendall’s  tau-b  statistical  test  indicates  there  is  a  moderate  positive 
relationship  between  the  rankings  of  the  two  variables,  and  that  the  relationship  is  statistically 
significant  at  95%  confidence. 


Q34  Do  your  subordinates'  performance  objectives  support  your  own  performance 
objectives?  *  Q35  Do  your  subordinates'  performance  objectives  inciude  measurabie 


standards? 

Do  your  subordinates'  performance 
objectives  support  your  own 
performance  objectives? 

Always  /  Often 

Sometimes  /  Rarely 

Do  your  subordinates'  performance 
objectives  include  measurable 
standards? 

Always  / 
Often 

67% 

48% 

Sometimes  / 
Rarely 

33% 

52% 

Total 

100% 

100% 

(16) 


Q34  Do  your  subordinates'  performance  objectives  support  your  own  performance 
objectives?  *  Q35  Do  your  subordinates'  performance  objectives  inciude  measurabie 

standards? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal  Kendall’s  tau-b 

Nominal  by  Nominal  Cramer’s  V 

0.241 

0.279 

0.044 

0.148 

(17) 


Three-quarters  of  respondents  (74.6%)  consistently  set  performance  objectives  for  their 
subordinates;  however,  only  59%  consistently  include  measurable  standards. 


Q32  Do  you  set  performance  objectives  for  your  immediate  subordinates? 

Frequency 

Percent 

Valid  Percent 

Valid  Always 

22 

34.9 

37.3 

Often 

22 

34.9 

37.3 

Sometimes 

11 

17.5 

18.6 

Rarely 

4 

6.3 

6.8 

Total 

59 

93.7 

100.0 
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Q35  Do  your  subordinates'  performance  objectives  inciude  measurabie 

standards? 

Frequency 

Percent 

Valid  Percent 

Always 

5 

7.9 

8.5 

Often 

30 

47.6 

50.8 

Valid  Sometimes 

18 

28.6 

30.5 

Rarely 

6 

9.5 

10.2 

Total 

59 

93.7 

100.0 

The  most  prevalent  method  for  setting  subordinates’  performance  objectives  is  based  on  the 
manager’s  own  superior’s  expectations  of  the  unit  (69.5%). 


Q33  How  do  you  set  performance  objectives  for  subordinates? 

Yes  (%) 

No  (%) 

Based  on  my  own  superior's  expectations  of  my  unit 

69.5 

30.5 

Based  on  subordinate' s  PLP 

64.4 

35.6 

Based  on  corporate  initiatives 

62.7 

37.3 

Based  on  work  description  or  statement  of  duties 

55.9 

44.1 

Based  on  past  years'  performance 

54.2 

45.8 

Based  on  milestones  in  CPME 

50.8 

49.2 

Based  on  other  criteria 

11.9 

88.1 

Nearly  twelve  percent  (1 1.9%)  also  base  objectives  on  ‘other  criteria’  which  may  include: 
career  requirements  for  advancement; 
pay  plan; 
subordinate  input; 

objectives  for  overall  Section  performance  improvement;  and, 
documentation  such  as  terms  of  reference,  goals  and  objectives. 

Almost  80%  of  Level  3  respondents  say  they  make  reports  of  progress  to  their  superior  during  a 
formal  program  review;  17.5%  say  they  do  not  make  reports  of  progress,  while  3.2%  are  unsure. 


Q28  Do  you  make  progress  reports  of  work  to  your  superior  during  a  formai 

program  review? 

Frequency 

Valid  Percent 

Yes 

50 

79.4 

.  No 

11 

17.5 

Valid  , ,  ^ 

Uncertain/  Don  t  Know 

2 

3.2 

Total 

63 

100.0 
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The  results  of  question  28  were  cross-tabulated  against  centre  to  further  investigate  the  13  ‘No’  or 
‘Uncertain/Don’t  know’  responses.  Six  of  the  13  responses  came  from  S&T  Operations  and  COS 
ADM(S&T)  where  annual  internal  program  reviews  are  not  held.  The  remainder  came  from 
DRDC’s  Valcartier,  Toronto  and  Suffield,  where  there  were  a  total  of  25  ‘yes’  responses.  There  are 
a  number  of  possible  reasons  for  the  ‘No’  responses,  such  as:  the  respondents  could  be  Chief 
Scientists  or  DDGs,  who  typically  support  the  DG  during  program  reviews,  rather  than  reporting; 
the  respondents  could  be  managers  who  are  for  some  reason  not  expected  to  present  at  centre 
program  reviews;  or,  the  respondents  could  be  new  Level  3s  who  were  not  yet  aware  of  that 
requirement. 


Q28  Do  you  make  progress  reports  of  work  to  your  superior 
during  a  formai  program  review?  *  Q1  Centre  Location 

Yes 

No 

Uncertain/ 
Don't  Know 

Valcartier,  Toronto,  Suffield, 
S&T  Operations,  Chief  of  Staff 

27 

11 

2 

The  number  of  Level  3  managers  who  report  requiring  reports  of  progress  from  subordinates  in 
preparation  for  a  formal  program  review  is  82%,  a  number  which  mirrors  responses  from  those  who 
say  they  make  reports  of  progress  to  their  own  superior  during  a  formal  program  review. 


Q30  Do  you  require  reports  of  progress  from  subordinates  in 
preparation  for  a  formai  program  review? 

Frequency 

Valid  Percent 

Yes 

41 

82.0 

Valid  No 

9 

18.0 

Total 

50 

100.0 
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It  appears  that  managers  who  are  required  to  make  progress  reports  to  superiors  in  preparation  for 
formal  reviews  are  more  likely  to  require  reports  of  progress  from  their  subordinates  in  preparation 
for  a  program  review.  Strictly  comparing  only  responses  of  ‘yes’  or  ‘no’  in  the  two  categories, 
Fisher’s  Exact  and  Phi  tests  suggest  there  is  a  strong  positive  relationship  between  the  variables, 
which  is  statistically  significant  at  95%  confidence. 


Q28  Do  you  require  reports  of  progress  from  subordinates  in  preparation  for  a  formai  program 
review?  *  Q30  Do  you  make  progress  reports  of  work  to  your  superior  during  a  formai  program 

review? 

Do  you  make  progress  reports  of 
work  to  your  superior  during  a 
formal  program  review? 

Yes 

No 

Total 

Do  you  require  reports  of  progress  from  yes 

35 

5 

40 

subordinates  in  preparation  for  a  formal 
program  review?  No 

4 

5 

9 

Total 

39 

10 

49 

(24) 


Q30  Do  you  require  reports  of  progress  from  subordinates  in  preparation  for  a 
formai  program  review?  *  Q28  Do  you  make  progress  reports  of  work  to  your 

superior  during  a  formai  program  review? 

Value 

Approx  Sig. 

Exact  Sig. 

Fisher’s  Exact  Test 

n/a 

n/a 

0.011 

Phi 

0.414 

0.004 

0.011 

(25) 


Level  3  managers  were  asked  to  describe,  in  general,  how  they  deal  with  situations  where 
subordinates  fail  to  meet  set  performance  objectives  (Q38). 

Themes  arising  from  these  comments  are  positive  for  the  most  part.  The  most  prevalent  method  of 
dealing  with  subordinates  who  fail  to  meet  objectives  is  to  discuss  the  problem  face  to  face,  identify 
the  core  issues,  collaborate  to  find  solutions,  develop  a  revised  performance  plan  and  then  monitor 
progress. 

“Mutual  agreement  of  the  problem  at  hand,  mutual  agreement  on  a  corrective  plan,  including 
specific  timelines  to  correct  the  situation,  and  regular  follow  up  till  satisfactory  resolution.” 

Solutions  to  objective  failures  may  include  reassigning  some  of  the  subordinate’s  other  duties  so 
s/he  can  devote  more  resources  to  the  project,  adding  another  subordinate  to  the  project,  or  entirely 
re -assigning  the  project  to  another  individual.  Some  managers  also  report  utilizing  HR  systems; 
initially  this  may  include  coaching  and  formal  training,  and  may  take  on  more  corrective 
applications  such  as  successive  disciplinary  action,  where  necessary.  Some  managers  report  noting 
project  failures  for  discussion  at  annual  employee  performance  reviews  and/or  when  developing 
Personal  Learning  Plans. 

“Work  out  a  solution  to  assist  subordinates  to  achieve  the  performance  objectives  that  have  been 
set.  This  could  be  accomplished  through  learning  and  development;  work  re-assignment; 
coaching,  etc.” 
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“Comments  on  PER.” 

“Modify  the  individual's  Learning  Plan.” 

Managers  tend  to  differentiate  between  failures  due  to  factors  beyond  the  subordinate’s  control,  and 
factors  within  their  control. 

“If  failure  to  meet  [objectives]  is  a  result  of  technical  challenges  or  circumstances  outside  of 
individual  control  (i.e.  where  sufficient  planning  and  effort  have  occurred),  then  1  help  them 
plan  altematives/corrective  measures.  We  increase  driving  forces  and  reduce  restraining 
forces.” 

“If  failure  to  meet  is  a  result  of  quality/quantity  of  effort  put  forth  by  the  individual,  then  I  counsel 
them,  set  metrics  that  are  smaller  and  closer  together,  and  work  with  them.” 

Other  themes  emerging  from  the  comments  appear  to  be  of  questionable  value.  Some  managers 
report  they  address  failures  by  simply  re -iterating  the  importance  of  successful  project  completion. 
Examples  of  this  type  of  comment  are: 

“Keep  asking  until  it  gets  done.” 

“Negotiate,  nag,  and  (rarefy)  issue  direct  orders  to  get  on  with  it.” 

Some  managers  also  report  dealing  with  failure  to  meet  objectives  by  revising  the  objectives.  If  this 
is  done  as  a  result  of  a  reassessment  of  resources  and  constraining  factors,  it  may  be  reasonable; 
however,  some  comments  seem  to  suggest  that  objectives  are  revised  as  a  way  of  making  the  project 
appear  successful. 

“Either  1  repeat  what  my  expectations  are  or  1  change  my  expectations.” 

Accountability:  Duties  and  Tasks 

The  survey  addressed  tasks  and  duties  separately.  Tasks  are  specific  work  items  assigned  by  a 
superior  that  could  be  associated  with  milestones,  whereas  duties  represent  ongoing  responsibilities 
that  would  include  delegated  authorities  under  legislation,  such  as  FAA.  In  the  language  of  the 
MAE,  tasks  relate  to  results  while  duties  relate  to  due  process. 

Half  (50.8%)  of  Level  3  respondents  acknowledge  a  distinction  between  assigned  tasks  and  ongoing 
duties  in  their  own  work  (Q27).  Respondents  are  somewhat  more  consistent  (61%)  in  making  a 
distinction  between  tasks  and  duties  when  they  assign  them  to  subordinates  (Q3 1). 


Q27  Do  you  see  a  distinction  between  ongoing  duties  and  specific 
tasks  assigned  to  you  by  your  superior? 

Frequency 

Percent 

Valid  Percent 

Yes 

32 

50.8 

50.8 

No 

28 

44.4 

44.4 

Uncertain/  Don't 
Know 

3 

4.8 

4.8 

Total 

63 

100.0 

100.0 
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Q31  Do  you  distinguish  between  duties  and  tasks  that  you  assign 
to  subordinates? 

Frequency 

Percent 

Valid  Percent 

Valid 

Always 

9 

14.3 

15.3 

Often 

27 

42.9 

45.8 

Sometimes 

16 

25.4 

27.1 

Rarely 

6 

9.5 

10.2 

Never 

1 

1.6 

1.7 

Total 

59 

93.7 

100.0 

Nearly  70%  of  those  who  distinguish  between  their  own  duties  and  tasks  also  distinguish  between 
the  duties  and  tasks  of  their  subordinates;  conversely,  nearly  the  same  percentage  who  do  not 
distinguish  between  their  own  duties  and  tasks,  also  do  not  distinguish  between  the  duties  and  tasks 
of  their  subordinates.  Results  of  a  tau-b  statistical  test  show  a  moderate-strong  positive  relationship 
between  the  variables,  which  is  significant  at  99%  confidence.  This  suggests  that  the  manager  who 
does  see  a  distinction  between  duties  and  tasks  assigned  by  a  superior  is  also  more  apt  to  be 
consistent  in  distinguishing  between  duties  and  tasks  that  are  assigned  to  subordinates. 


Q31  Do  you  distinguish  between  duties  and  tasks  that  you  assign  to  subordinates?  *  Q27  Do  you 
see  a  distinction  between  ongoing  duties  and  specific  tasks  assigned  to  you  by  your  superior? 

Do  you  see  a  distinction  between  ongoing  duties 
and  specific  tasks  assigned  to  you  by  your 
superior? 

Yes 

No 

Uncertain/ 
Don't  Know 

Total 

Do  you  distinguish  between  duties 

Always  / 
Often 

25 

9 

2 

36 

and  tasks  that  you  assign  to 
subordinates? 

Sometimes 
/  Rarely 

6 

15 

1 

22 

Never 

0 

1 

0 

1 

Total 

31 

25 

3 

59 

(28) 


Q31  Do  you  distinguish  between  duties  and  tasks  that  you  assign  to 
subordinates?  *  Q27  Do  you  see  a  distinction  between  ongoing  duties  and 
specific  tasks  assigned  to  you  by  your  superior? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal  Kendall’s  tau-b 

0.381 

0.001 

Nominal  by  Nominal  Cramer’s  V 

0.319 

0.017 

(29) 
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Level  3  Managers  communicate  more  often  on  a  weekly  basis  with  subordinates  about  assigned 
tasks  than  about  ongoing  duties. 


Q36,  Q  37  How  often  do  you  communicate  with  your  subordinates 
about  ongoing  duties  and  assigned  tasks? 

Weekly 

Monthly 

2-  4  Times/ 
Year 

Ongoing  Duties 

33.9  % 

47.5  % 

18.6  % 

Assigned  Tasks 

54.2  % 

40.7  % 

5.1  % 

(30) 

Level  3  managers  who  are  inconsistent  in  differentiating  between  duties  and  tasks  appear  slightly 
more  likely  to  communicate  on  a  weekly  and  monthly  basis  with  subordinates  to  discuss  duties  than 
managers  who  are  consistent  in  distinguishing  between  tasks  and  duties.  However,  statistical  testing 
shows  a  very  weak  negative  relationship  between  the  variables,  which  is  not  statistically  significant. 


Q36  How  often  do  you  communicate  with  your  subordinates  about  ongoing  duties?  *  Q31  Do 

you  distinguish  between  duties  and  tasks  that  you  assign  to  subordinates? 

Do  you  distinguish  between  duties  and  tasks  that  you  assign  to 

subordinates? 

Always/  Often 

Sometimes  /  Rarely 

Never 

Weekly 

12 

33.3% 

8 

36.4% 

0 

0% 

How  often  do  you 
communicate  with 
your  subordinates  Monthly 

16 

44.5% 

11 

50.0% 

1 

100% 

about  ongoing 

duties?  2-4  Times  per 

Year 

8 

22.2% 

3 

13.6% 

0 

0% 

Total 

36 

100% 

22 

100% 

1 

100% 

(31) 


Q36  How  often  do  you  communicate  with  your  subordinates  about 
ongoing  duties?  *  Q31  Do  you  distinguish  between  duties  and  tasks  that 

you  assign  to  subordinates? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal  Kendall’s  tau-c 

-0.067 

0.568 

Nominal  by  Nominal  Cramer’s  V 

0.217 

0.698 

(32) 
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Level  3  managers  who  are  inconsistent  in  differentiating  between  duties  and  tasks  also  appear 
slightly  more  likely  to  communicate  on  a  weekly  basis  with  subordinates  to  discuss  assigned  tasks 
than  managers  who  are  consistent  in  distinguishing  between  tasks  and  duties.  However,  statistical 
testing  shows  a  very  weak  negative  relationship  between  the  variables,  which  is  not  statistically 
significant. 


Q37  How  often  do  you  communicate  with  your  subordinates  about  assigned  tasks?  * 

Q31  Do  you 

distinguish  between  duties  and  tasks  that  you  assign  to  subordinates? 

Do  you  distinguish  between  duties  and  tasks  that  you  assign 

to  subordinates? 

Always/  Often 

Sometimes/  Rarely 

Never 

How  often  do  you  Weekly 

18 

50% 

13 

59% 

1 

100% 

communicate  with 

16 

44% 

8 

36% 

0 

0% 

your  subordinates 

about  assigned  „  ,  .... 

tasks?  2-4  Times  per 

Year 

2 

6% 

1 

5% 

0 

0% 

Total 

36 

100% 

22 

100% 

1 

100% 

(33) 


Q37  How  often  do  you  communicate  with  your  subordinates  about 
assigned  tasks?  *  Q31  Do  you  distinguish  between  duties  and  tasks  that 

you  assign  to  subordinates? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal  Kendall’s  tau-c 

-0.033 

0.756 

Nominal  by  Nominal  Cramer’s  V 

0.247 

0.513 

Managers  who  distinguish  between  the  duties  and  tasks  assigned  to  subordinates  appear  more  likely 
to  consistently  include  measurable  standards  in  the  performance  objectives  of  subordinates. 


Q35  Do  your  subordinates'  performance  objectives  inciude  measurabie 
standards?  *  Q31  Do  you  distinguish  between  duties  and  tasks  that  you  assign 

to  subordinates? 

Do  you  distinguish  between  duties 

and  tasks  that  you  assign  to 

subordinates? 

Always  /  Often 

Sometimes  / 
Rarely 

Do  your  subordinates' 

Always  /  Often 

67% 

45% 

performance  objectives 

include  measurable 

Sometimes  / 

standards? 

Rarely 

33% 

55% 

Total 

100% 

100% 

(35) 
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A  non-parametric  Kendall’s  tau-b  test  indicates  there  is  a  moderate  positive  relationship  between  the 
rankings  of  the  two  variables,  and  that  the  relationship  is  statistically  significant  at  95%  confidence. 


Q35  Do  your  subordinates'  performance  objectives  inciude  measurabie 
standards?  *  Q31  Do  you  distinguish  between  duties  and  tasks  that  you 
assign  to  subordinates? 


Value 

Approx.  Sig. 

Ordinal  by  Ordinal 

Kendall’s  tau-b 

0.270 

0.023 

Nominal  by  Nominal 

Cramer’s  V 

0.262 

0.215 

(36) 


Risk  Management  Training 

There  are  both  positive  and  negative  results  for  Risk  Management.  Despite  only  6%  of  Level  3 
managers  reporting  having  received  risk  management  training  in  the  last  two  fiscal  years,  nearly  3/4 
of  Level  3  managers  (7 1  %)  believe  they  understand  their  role  in  RM,  almost  2/3  (62%)  believe  they 
understand  their  responsibilities  for  RM,  and  more  than  half  (56%)  believe  they  understand  what 
their  authorities  are  in  RM. 


Q15  Did  you  receive  risk  management  training  in  this  fiscai  year  or 

iast? 

Frequency 

Valid  Percent 

Yes 

4 

6.3 

No 

59 

93.7 

Total 

63 

100.0 

Q17  Do  you  understand  your  role  in  managing  risk?  *  Q15  Did  you  receive  risk  management 

training  in  this  fiscal  year  or  last? 

Did  you  receive  risk  management 
training  in  this  fiscal  year  or  last? 

Yes 

No 

Total 

Do  you  understand  your  role  in 

Yes 

4 

41 

45 

managing  risk? 

No 

Uncertain/  Don't 

0 

4 

4 

Know 

0 

14 

14 

Total 

4 

59 

63 

(38) 
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Q18  Do  you  understand  your  responsibilities  for  managing  risk?  *  Q15  Did  you  receive  risk 
management  training  in  this  fiscal  year  or  last? 

Did  you  receive  risk  management 
training  in  this  fiscal  year  or  last? 

Yes 

No 

Total 

Do  you  understand  your 

Yes 

3 

36 

39 

responsibilities  for  managing 

No 

0 

5 

5 

risk? 

Uncertain/  Don't 
Know 

1 

18 

19 

Total 

4 

59 

63 

(39) 


Q19  Do  you  understand  the  limitations  on  your  authority  for  managing  risk?  *  Q15  Did  you 
receive  risk  management  training  in  this  fiscal  year  or  last? 

Did  you  receive  risk  management 
training  in  this  fiscal  year  or  last? 

Total 

Yes 

No 

Do  you  understand  your  Yes 

2 

33 

35 

authority  for  managing  risk?  nq 

1 

8 

9 

Uncertain/  Don't 

Know 

1 

18 

19 

Total 

4 

59 

63 

(40) 


On  the  other  hand,  a  significant  number  of  Level  3  managers  are  uncertain  what  their  roles, 
responsibilities  and  authorities  are  in  relation  to  risk  management. 


Don't  Know/ 
Uncertain 

Q  17  Do  you  understand  your  role  in 
managing  risk? 

22% 

Q18  Do  you  understand  your 
responsibilities  for  managing  risk? 

30% 

Q19  Do  you  understand  the 
limitations  on  your  authority  for 
managing  risk? 

30% 

(41) 
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Nearly  80%  of  managers  report  that  no  subordinates  in  their  unit  have  received  risk  management 
training  in  the  last  two  fiscal  years. 


Q39  How  many  of  your  subordinates  received  risk 
management  training  this  fiscai  year  or  iast? 

Frequency 

Valid 

Percent 

Valid  Some 

12 

20.3 

None 

47 

79.7 

Total 

59 

100.0 

Q15  Did  you  receive  risk  management  training  in  this  fiscai  year  or  iast?  *  Q39 
How  many  of  your  subordinates  received  risk  management  training  this  fiscai 

year  or  iast? 

How  many  of  your  subordinates 
received  risk  management 
training  this  fiscal  year  or  last? 

Total 

Some 

None 

Did  you  receive  yes 

4 

0 

4 

risk  management 

training  in  this 

fiscal  year  or  last?  No 

8 

47 

55 

Total 

12 

47 

59 

(43) 


DRDC  Atlantic  CR  2009-135 


117 


Risk  Management  and  Planning 

Risk  items  do  not  appear  to  be  properly  or  consistently  documented  as  a  part  of  project  management. 
Eighty  percent  of  respondents  record  risk  items  in  project  documentation,  37%  record  risk  items  in 
personal  documentation,  14  %  of  respondents  record  risk  items  in  CPME,  and  14%  do  not  record 
risk  items.  Eleven  percent  document  risk  items  elsewhere,  such  as  Quad  Charts,  TAG  and  PG 
meetings  minutes,  and  SRB  presentations.  Two  respondents  note  that  in  their  experience,  risk 
documentation  only  occurs  for  TDP  projects,  not  for  ARP  or  other  projects. 


Q25  Where  are  identified  project  risks  documented? 

yes  (%) 

no  (%) 

project  documentation 

79.6 

20.4 

personal  documentation 

37.0 

63.0 

CPME  database 

14.8 

85.2 

risks  are  not  documented 

14.8 

85.2 

elsewhere 

11.1 

88.9 

Roughly  one  third  (35.9%)  of  E3  respondents  report  ‘Always’  or  ‘Often’  identifying  risk  items  in  a 
likelihood/  impact  matrix. 


Q23  Do  project  plans  explicitly  identify  specific  risks  in  terms  of 
both  likelihood  and  impact? 

Frequency 

Percent 

Valid  Percent 

Valid  Always 

3 

4.8 

5.7 

Often 

16 

25.4 

30.2 

Sometimes 

22 

34.9 

41.5 

Rarely 

10 

15.9 

18.9 

Never 

2 

3.2 

3.8 

Total 

53 

84.1 

100.0 
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Follow  up  on  risk  issues  during  projects  is  also  lacking.  Only  36%  of  respondents  report  revisiting 
project  risk  assessments  consistently,  while  52%  report  consistently  assessing  project  progress 
reports  against  organizational  plans  such  as  milestones  in  CPME. 


Q24  Are  project  risk  assessments  revisited  during  the  deiivery  of 

projects? 

Frequency 

Percent 

Valid  Percent 

Valid  Always 

2 

3.2 

4.0 

Often 

16 

25.4 

32.0 

Sometimes 

15 

23.8 

30.0 

Rarely 

15 

23.8 

30.0 

Never 

2 

3.2 

4.0 

Total 

50 

79.4 

100.0 

Managers  who  report  that  project  plans  consistently  identify  the  likelihood  and  impact  of  risks 
appear  more  likely  to  revisit  risk  assessments  during  the  delivery  of  the  project.  Statistical  tests 
show  a  very  strong  positive  relationship  between  the  two  variables,  which  is  statistically  significant 
at  99.9%  confidence.  This  suggests  that  those  who  explicitly  identify  project  risks  through  a 
likelihood/  impact  matrix  are  more  likely  to  revisit  risk  assessments  during  the  delivery  of  that 
project. 


Q23  Do  project  pians  expiicitiy  identify  specific  risks  in  terms  of  both  iikeiihood  and  impact?  * 
Q24  Are  project  risk  assessments  revisited  during  the  deiivery  of  projects? 

Do  project  plans  explicitly  identify  specific  risks  in  terms  of 
both  likelihood  and  impact? 

Always 

Often 

Sometimes 

Rarely 

Never 

Total 

Are  project  risk  Always 

assessments  Often 

revisited  during  sometimes 

the  delivery  of 

2 

1 

0 

0 

11 

4 

0 

4 

10 

0 

0 

1 

0 

0 

0 

2 

16 

15 

projects?  Rarely 

0 

0 

6 

8 

1 

15 

Never 

0 

0 

0 

1 

1 

2 

Total 

3 

15 

20 

10 

2 

50 

(47) 


Q23  Do  project  plans  explicitly  identify  specific  risks  in  terms  of 
both  likelihood  and  impact?  *  Q24  Are  project  risk  assessments 
revisited  during  the  delivery  of  projects? 

Value 

Approx. 

Sig. 

Ordinal  by  Ordinal  Kendall's  tau-b 

Nominal  by  Nominal  Cramer’s  V 

0.743 

0.619 

0.000 

0.000 

(48) 
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Little  more  than  half  (52%)  of  respondents  who  are  responsible  for  managing  projects  report 
consistently  comparing  progress  to  established  project  milestones. 


Q22  Do  you  compare  reports  of  progress  on  projects  to 
organizational  plans  (i.e.:  milestones  in  CPME)? 

Frequency 

Percent 

Valid 

Percent 

Valid  Always 

7 

11.1 

13.5 

Often 

20 

31.7 

38.5 

Sometimes 

15 

23.8 

28.8 

Rarely 

7 

11.1 

13.5 

Never 

3 

4.8 

5.8 

Total 

52 

82.5 

100.0 

Mangers  who  report  always  or  often  comparing  progress  to  established  project  milestones  also 
appear  somewhat  more  likely  to  report  always  or  often  identifying  the  likelihood  and  impact  of 
project  risks.  There  is  a  moderate  positive  relationship  between  these  variables,  which  is  significant 
at  95%  confidence. 


Q22  Do  you  compare  reports  of  progress  on  projects  to  organizational  plans  (i.e.:  milestones  in 
CPME)?  *  Q23  Do  project  plans  explicitly  identify  specific  risks  in  terms  of  both  likelihood  and 

impact? 


Do  project  plans  explicitly  identify  specific  risks  in  terms 

of  both  likelihood  and  im 

cact? 

Always 

Often 

Sometimes 

Rarely 

Never 

Total 

Do  you  compare 

Always 

3 

2 

2 

0 

0 

7 

reports  of  progress  on 

Often 

0 

8 

8 

3 

1 

20 

projects  to 
organizational  plans 

Sometimes 

0 

4 

6 

5 

0 

15 

(i.e.:  milestones  in 

Rarely 

0 

2 

4 

0 

1 

7 

CPME)? 

Never 

0 

0 

2 

1 

0 

3 

Total 

3 

16 

22 

9 

2 

52 

(50) 


Q22  Do  you  compare  reports  of  progress  on  projects  to 
organizational  plans  (i.e.:  milestones  in  CPME)?  *  Q23  Do  project 
plans  explicitly  identify  specific  risks  in  terms  of  both  likelihood 

and  impact? 

Value 

Approx. 

Sig. 

Ordinal  by  Ordinal  Kendall's  tau-b 

Nominal  by  Nominal  Cramer’s  V 

0.292 

0.381 

0.010 

0.017 
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Respondents  who  are  consistent  in  comparing  project  progress  reports  to  planned  milestones  appear 
to  be  somewhat  more  likely  to  consistently  revisit  project  risk  assessments  during  project  delivery. 
There  is  a  moderate  positive  relationship  between  the  two  variables,  which  is  significant  at  95% 
confidence. 


Q22  Do  you  compare  reports  of  progress  on  projects  to  organizational  plans  (i.e.:  milestones  in 

CPME)?‘ 

Q24  Are  project  risk  assessments  revisited  during  the  delivery  of  projects? 

Do  you  compare  reports  of  progress  on  projects  to 
organizational  plans  (i.e.:  milestones  in  CPME)? 

Always 

Often 

Sometimes 

Rarely 

Never 

Total 

Are  project  risk 
assessments 

Always 

2 

0 

0 

0 

0 

2 

revisited  during 

Often 

4 

6 

3 

2 

1 

16 

the  delivery  of 

Sometimes 

0 

6 

6 

2 

1 

15 

projects? 

Rarely 

0 

6 

5 

2 

1 

14 

Never 

0 

0 

1 

1 

0 

2 

Total 

6 

18 

15 

7 

3 

49 

(52) 


Q22  Do  you  compare  reports  of  progress  on  projects  to 
organizational  plans  (i.e.:  milestones  in  CPME)?  *  Q24  Are  project  risk 
assessments  revisited  during  the  delivery  of  projects? 

Value 

Approx. 

Sig. 

Ordinal  by  Ordinal  Kendall's  tau-b 

0.299 

0.016 

Nominal  by  Nominal  Cramer’s  V 

0.355 

0.075 

(53) 
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Risk  Management  Communication 

A  large  number  of  Level  3  managers  (46%)  responded  ‘Don’t  Know/  Uncertain’  when  asked  if  RM 
is  discussed  at  TAG  meetings;  this  may  be  due  to  a  respondent  not  being  a  participant  in  Thrust 
Advisory  Group  meetings. 


Q20  How  often  is  risk  management  discussed  at  TAG  meetings? 

Valid 

Cumulative 

Frequency 

Percent 

Percent 

Percent 

Valid  Always 

1 

1.6 

1.6 

1.6 

Often 

2 

3.2 

3.2 

4.8 

Sometimes 

14 

22.2 

22.2 

27.0 

Rarely 

12 

19.0 

19.0 

46.0 

Never 

5 

7.9 

7.9 

54.0 

Uncertain/  Don't 
know 

29 

46.0 

46.0 

100.0 

Total 

63 

100.0 

100.0 

(54) 


Removing  those  who  responded  ‘Don’t  Know/  Uncertain’,  Level  3  managers  report  that  risk 
management  is  inconsistently  (76.5%)  or  never  (14.7%)  discussed  at  TAG  meetings. 


Q20  How  often  is  risk  management  discussed  at  TAG  meetings? 

Frequency 

Percent 

Valid  Always 

1 

2.9 

Often 

2 

5.9 

Sometimes 

14 

41.2 

Rarely 

12 

35.3 

Never 

5 

14.7 

Total 

34 

100.0 

Barely  30%  of  respondents  report  being  consulted  consistently  for  risk  management  input  on 
projects  the  outcome  of  which  they  have  a  direct  or  related  stake. 


Q16  How  often  are  you  consulted  for  risk  management  input  on  projects  the 
outcome  of  which  you  have  a  direct  or  related  stake? 

Frequency 

Percent 

Valid  Percent 

Always 

3 

4.8 

4.8 

Often 

16 

25.4 

25.4 

Sometimes 
Valid  „  , 

Rarely 

26 

41.3 

41.3 

14 

22.2 

22.2 

Never 

4 

6.3 

6.3 

Total 

63 

100.0 

100.0 

122 


DRDC  Atlantic  CR  2009-135 


It  appears  that  the  main  forum  for  consultation  about  risk  issues  related  to  projects  is  TAG  meetings. 
A  comparison  of  the  variables  suggests  that  there  is  a  strong  relationship  between  respondents  who 
report  being  consulted  ‘Always’  or  ‘Often  ‘for  risk  management  input  on  projects  where  they  are 
stakeholders,  also  report  that  risk  issues  are  discussed  ‘Always’  or  ‘Often’  at  TAG  meetings.  This 
relationship  is  significant  at  99.9%  confidence. 


Q16  How  often  are  you  consulted  for  risk  management  input  on  projects  the  outcome  of  which  you 

have  a  direct  or  related  stake?  *  Q20  How  often  is  risk  management  discussed  at  TAG  meetings? 

How  often  is  risk  management  discussed  at  TAG 

meetings? 

Always 

Often 

Sometimes 

Rarely 

Never 

Total 

How  often  are  you 

Always 

0 

0 

1 

0 

0 

1 

consulted  for  risk 
management  input 

Often 

1 

1 

5 

3 

0 

10 

on  projects  the 

Sometimes 

0 

1 

8 

5 

2 

16 

outcome  of  which 
you  have  a  direct 

Rarely 

0 

0 

0 

4 

2 

6 

or  related  stake? 

Never 

0 

0 

0 

0 

1 

1 

Total 

1 

2 

14 

12 

5 

34 

(57) 


Q16  How  often  are  you  consulted  for  risk  management  input  on 
projects  the  outcome  of  which  you  have  a  direct  or  related  stake?  * 
Q20  How  often  is  risk  management  discussed  at  TAG  meetings? 

Value 

Approx. 

Sig. 

Ordinal  by  Ordinal  Kendall's  tau-b 

0.481 

0.000 

Nominal  by  Nominal  Cramer’s  V 

0.363 

0.327 

(58) 

According  to  Level  3  respondents,  identified  project  stakeholders  are  most  often  consulted  about 
risk  issues  in  projects  during  the  planning  phase  (86.7%);  other  times  include  during  project  reviews 
(77.8%)  and  as  risk  related  events  occur  (77.8%).  There  are  no  statistical  relationships  between 
when  stakeholders  are  consulted  (Q26)  and  Risk  Management  discussions  at  TAG  meetings  (Q20). 


Q26  When  are  project  stakeholders  consulted  about  risk  issues 
that  may  affect  them? 

Yes  (%) 

No  (%) 

When  planning  a  project 

86.7 

13.3 

During  a  scheduled  project  review 

77.8 

22.2 

As  risk-related  events  occur 

77.8 

22.2 

Other 

4.4 

95.6 

Never 

0.0 

100.0 

(59) 
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Overall,  Managers  are  inconsistent  in  communicating  risk  tolerances  and  priorities  to  subordinates; 
only  27%  report  consistently  making  their  risk  priorities  and  tolerances  clear  to  subordinates. 


Q40  Do  you  make  risk  management  priorities  and 
toierances  ciear  to  subordinates? 

Frequency 

Valid 

Percent 

Valid  Always 

2 

3.4 

Often 

14 

23.7 

Sometimes 

31 

52.5 

Rarely 

10 

16.9 

Never 

2 

3.4 

Total 

59 

100.0 

Level  3  Managers  do  not  appear  to  consistently  have  confidence  that  their  subordinates  understand 
their  roles,  responsibilities,  or  limitations  on  authority  when  managing  risks.  In  each  case,  over  half 
of  respondents  either  have  inconsistent  or  no  confidence  that  their  subordinates  understand  these  risk 
management  duties. 


Q41  Do  you  have  confidence  your  subordinates 
understand  their  roies  for  managing  risks? 

Frequency 

Valid  Percent 

Always 

2 

3.4 

Often 

25 

42.4 

Sometimes 

20 

33.9 

Valid 

Rarely 

10 

16.9 

Never 

2 

3.4 

Total 

59 

100.0 

(61) 


Q42  Do  you  have  confidence  your  subordinates 
understand  their  responsibiiities  for  managing  risks? 

Frequency 

Valid  Percent 

Always 

1 

1.7 

Often 

28 

47.5 

Sometimes 

20 

33.9 

Valid 

Rarely 

9 

15.3 

Never 

1 

1.7 

Total 

59 

100.0 

(62) 
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Q43  Do  you  have  confidence  your  subordinates 
understand  the  iimitations  on  their  authority  for 
managing  risks? 

Frequency 

Valid  Percent 

Always 

5 

8.5 

Often 

23 

39.0 

Sometimes 

15 

25.4 

Valid 

Rarely 

13 

22.0 

Never 

3 

5.1 

Total 

59 

100.0 

Not  only  do  subordinates  fail  to  understand  the  scope  of  their  authority  in  risk  management,  many 
Level  3  Managers  believe  that  many  lack  the  required  authority  to  effectively  manage  risk. 


Q45  Do  subordinates  responsibie  for  managing  risks 
have  required  authority  to  do  so  effectiveiy? 

Frequency 

Valid  Percent 

All 

16 

27.1 

Some 

35 

59.3 

Valid 

None 

8 

13.6 

Total 

59 

100.0 

Statistical  testing  supports  the  assumption  that  managers  who  make  their  risk  management  priorities 
and  tolerances  clear  to  subordinates  are  more  likely  to  have  confidence  in  their  subordinates’ 
understanding  of  their  roles  in  managing  risks.  There  is  a  moderate -strong  positive  relationship 
between  the  variables,  which  is  statistically  significant  at  99%  confidence. 


Q41  Do  you  have  confidence  your  subordinates  understand  their  roies  for  managing  risks?  * 

Q40  Do  you  make  risk  management  priorities  and  toierances  ciear  to  subordinates? 

Do  you  make  risk  management  priorities  and 

tolerances  clear  to  subordinates? 

Always 

Often 

Sometimes 

Rarely 

Never 

Total 

Do  you  have  Always 

0 

1 

1 

0 

0 

2 

confidence  your  Often 

1 

10 

10 

4 

0 

25 

subordinates  Sometimes 

1 

1 

16 

2 

0 

20 

understand  their  Rarely 

0 

1 

4 

4 

1 

10 

roles  for  managing  Never 

risks? 

0 

1 

0 

0 

1 

2 

Total 

2 

14 

31 

10 

2 

59 

(65) 
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Q41  Do  you  have  confidence  your  subordinates  understand  their 
roies  for  managing  risks?  *  Q40  Do  you  make  risk  management 
priorities  and  toierances  ciear  to  subordinates? 

Value 

Approx. 

Sig. 

Ordinal  by  Ordinal  Kendall's  tau-b 

Nominal  by  Nominal  Cramer’s  V 

0.323 

0.376 

0.009 

0.006 

(66) 

Statistical  testing  supports  the  assumption  that  managers  who  make  their  risk  management  priorities 
and  tolerances  clear  to  subordinates  are  more  likely  to  have  confidence  in  their  subordinates’ 
understanding  of  their  responsibilities  in  managing  risks.  There  is  a  moderate-strong  positive 
relationship  between  the  variables,  which  is  statistically  significant  at  99%  confidence. 


Q42  Do  you  have  confidence  your  subordinates  understand  their  responsibiiities  for 
managing  risks?  *  Q40  Do  you  make  risk  management  priorities  and  toierances  ciearto 

subordinates? 


Do  you  make  risk  management  priorities  and 

tolerances  clear  to  subordinates? 

Always 

Often 

Sometimes 

Rarely 

Never 

Total 

Do  you  have 

Always 

0 

1 

0 

0 

0 

1 

confidence  your 

Often 

2 

9 

13 

4 

0 

28 

subordinates 

Sometimes 

0 

3 

15 

2 

0 

20 

understand  their 

Rarely 

0 

1 

3 

4 

1 

9 

responsibilities  for 
managing  risks? 

Never 

0 

0 

0 

0 

1 

1 

Total 

2 

14 

31 

10 

2 

59 

(67) 


Q42  Do  you  have  confidence  your  subordinates  understand  their 
responsibiiities  for  managing  risks?  *  Q40  Do  you  make  risk 
management  priorities  and  toierances  ciear  to  subordinates? 

Value 

Approx. 

Sig. 

Ordinal  by  Ordinal  Kendall's  tau-b 

Nominal  by  Nominal  Cramer’s  V 

0.373 

0.444 

0.002 

0.000 

(68) 
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Statistical  testing  supports  the  assumption  that  managers  who  make  their  risk  management  priorities 
and  tolerances  clear  to  subordinates  are  more  likely  to  have  confidence  in  their  subordinates’ 
understanding  of  their  authorities  in  managing  risks.  There  is  a  moderate -strong  positive  relationship 
between  the  variables,  which  is  statistically  significant  at  99%  confidence. 


Q43  Do  you  have  confidence  your  subordinates  understand  the  iimitations  on  their  authority  for 
managing  risks?  *  Q40  Do  you  make  risk  management  priorities  and  toierances  ciearto 

subordinates? 


Do  you  make  risk  management  priorities  and 

tolerances  clear  to  subordinates? 

Always 

Often 

Sometimes 

Rarely 

Never 

Total 

Do  you  have 

Always 

0 

4 

1 

0 

0 

5 

confidence  your 

Often 

2 

5 

12 

3 

1 

23 

subordinates 

Sometimes 

0 

3 

12 

0 

0 

15 

understand  the 

Rarely 

0 

1 

5 

6 

1 

13 

limitations  on  their 

Never 

authority  for 
managing  risks? 

0 

1 

1 

1 

0 

3 

Total 

2 

14 

31 

10 

2 

59 

(69) 


Q43  Do  you  have  confidence  your  subordinates  understand  the 
iimitations  on  their  authority  for  managing  risks?  *  Q40  Do  you  make 
risk  management  priorities  and  toierances  ciear  to  subordinates? 

Value 

Approx. 

Sig. 

Ordinal  by  Ordinal  Kendall's  tau-b 

Nominal  by  Nominal  Cramer’s  V 

0.332 

0.341 

0.004 

0.036 

(70) 

Subordinates  bring  risk  issues  to  the  attention  of  managers  with  varying  levels  of  frequency;  22%  of 
managers  have  issues  brought  to  them  weekly;  35.6%  monthly  and  33.9%  quarterly  or  semi¬ 
annually.  However,  this  does  not  give  any  information  about  the  risk  factor’s  importance  or  impact. 


Q44  How  often  do  subordinates  bring  risk  issues  to  your 
attention? 

Frequency 

Valid  Percent 

Weekly 

13 

22.0 

Monthly 

21 

35.6 

2-4  Times/ 
Valid  Year 

20 

33.9 

Annually 

3 

5.1 

Never 

2 

3.4 

Total 

59 

100.0 
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Managers  who  do  not  consistently  distinguish  between  tasks  and  duties  appear  to  be  nearly  twice  as 
likely  to  have  subordinates  bring  RM  issues  to  them  on  a  weekly  basis.  Again,  this  does  not  give  any 
information  about  the  significance  of  the  risk  factor’s  importance  or  impact.  However,  a  Kendall’s 
tau-b  test  indicates  there  is  no  significant  relationship  between  the  variables. 


Q31  Do  you  distinguish  between  duties  and  tasks  that  you  assign  to  subordinates?  *  Q44  How 
often  do  subordinates  bring  risk  issues  to  your  attention? 

How  often  do  subordinates  bring  risk  issues  to  your  attention? 

2-4 

Times 

Weekly  Monthly  per  Year  Annually  Never  Total 

Do  you 
distinguish 
between  duties 
and  tasks  that 
you  assign  to 
subordinates? 

Always  /  Often 

6  15  11  3  1  36 

17%  42%  31%  8%  2%  100% 

Sometimes/ 

Rarely 

7  6  8  0  1  22 

32%  27%  36%  0%  5%  100% 

Never 

0  0  1  0  0  1 

0%  0%  100%  0%  0%  100% 

Total 

13  21  20  3  2  59 

(72) 


Q31  Do  you  distinguish  between  duties  and  tasks  that  you  assign  to 
subordinates?  *  Q44  How  often  do  subordinates  bring  risk  issues  to 

your  attention? 

Value 

Approx. 

Sig. 

Ordinal  by  Ordinal 

Kendall's  tau-b 

-0.071 

0.573 

Nominal  by  Nominal 

Cramer’s  V 

0.264 

0.419 
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Only  1 7%  of  Level  3  managers  report  that  risk  management  is  consistently  (always  or  often) 
considered  when  assessing  subordinates’  performance;  73%  inconsistently  (sometimes  or  rarely) 
consider  subordinates  risk  management  performance  as  a  part  of  performance  assessment,  and  1 0% 
report  never  considering  risk  management  performance. 


Q46  Is  risk  management  given  consideration  when  assessing  the  performance  of  subordinates? 

Frequency 

Percent 

Valid 

Percent 

Cumulative 

Percent 

Valid 

Always 

1 

1.6 

1.7 

1.7 

Often 

9 

14.3 

15.3 

16.9 

Sometimes 

22 

34.9 

37.3 

54.2 

Rarely 

21 

33.3 

35.6 

89.8 

Never 

6 

9.5 

10.2 

100.0 

Total 

59 

93.7 

100.0 

Missing 

99 

4 

6.3 

Total 

63 

100.0 

(74) 


There  is  a  moderate -strong  positive  relationship  between  how  consistently  risk  management  is  given 
consideration  when  assessing  the  performance  of  subordinates  and  how  often  risk  issues  are  brought 
to  the  supervisor’s  attention.  This  relationship  is  statistically  significant  at  99.9%  confidence. 


Q44  How  often  do  subordinates  bring  risk  issues  to  your  attention?  *  Q46  is  risk  management 
given  consideration  when  assessing  the  performance  of  subordinates? 

Is  risk  management  given  consideration  when  assessing 
the  performance  of  subordinates? 

Always 

Often 

Sometimes 

Rarely 

Never 

Total 

How  often  do 

Weekly 

0 

3 

7 

3 

0 

13 

subordinates 
bring  risk 

Monthly 

0 

5 

9 

6 

1 

21 

issues  to  your 
attention? 

2-4  Times 
per  Year 

0 

1 

6 

10 

3 

20 

Annually 

1 

0 

0 

2 

0 

3 

Never 

0 

0 

0 

0 

2 

2 

Total 

1 

9 

22 

21 

6 

59 

(75) 


Q44  How  often  do  subordinates  bring  risk  issues  to  your  attention?  *  Q46 

Is  risk  management  given  consideration  when  assessing  the  performance 

of  subordinates? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal 

Kendall's  tau-b 

0.343 

0.001 

Nominal  by  Nominal 

Cramer’s  V 

0.450 

0.000 

(76) 
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There  is  a  moderate -strong  positive  relationship  between  managers’  ratings  of  their  confidence  in 
subordinates’  understanding  of  their  roles  for  managing  risks  and  their  ratings  of  how  consistently 
risk  management  is  given  consideration  when  assessing  the  performance  of  subordinates.  This 
relationship  is  statistically  significant  at  99.9%  confidence. 


Q41  Do  you  have  confidence  your  subordinates  understand  their  roies  for  managing  risks?  *  Q46 
is  risk  management  given  consideration  when  assessing  the  performance  of  subordinates? 


Is  risk  management  given  consideration  when 
assessing  the  performance  of  subordinates? 

Total 

Always 

Often 

Sometimes 

Rarely 

Never 

Do  you  have 

Always 

0 

0 

1 

1 

0 

2 

confidence  your 

Often 

1 

6 

13 

4 

1 

25 

subordinates 

Sometimes 

0 

2 

7 

9 

2 

20 

understand  their 

Rarely 

0 

1 

1 

6 

2 

10 

roles  for  managing 
risks? 

Never 

0 

0 

0 

1 

1 

2 

Total 

1 

9 

22 

21 

6 

59 

(77) 


Q41  Do  you  have  confidence  your  subordinates  understand  their  roies 
for  managing  risks?  *  Q46  is  risk  management  given  consideration 
when  assessing  the  performance  of  subordinates? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal  Kendall's  tau-b 

Nominal  by  Nominal  Cramer’s  V 

0.388 

0.279 

0.000 

0.302 

(78) 


There  is  a  strong  positive  relationship  between  managers’  ratings  of  their  confidence  in 
subordinates’  understanding  of  their  responsibilities  for  managing  risks  and  their  ratings  of  how 
consistently  risk  management  is  given  consideration  when  assessing  the  performance  of 
subordinates.  This  relationship  is  statistically  significant  at  99.9%  confidence. 


Q42  Do  you  have  confidence  your  subordinates  understand  their  responsibiiities  for  managing 
risks?  *  Q46  is  risk  management  given  consideration  when  assessing  the  performance  of 

subordinates? 

Is  risk  management  given  consideration  when 
assessing  the  performance  of  subordinates? 

Always 

Often 

Sometimes 

Rarely 

Never 

Total 

Do  you  have 
confidence  your 

Always 

0 

1 

0 

0 

0 

1 

subordinates 
understand  their 

Often 

1 

5 

15 

6 

1 

28 

responsibilities  for 

Sometimes 

0 

2 

6 

10 

2 

20 

managing  risks? 

Rarely 

0 

1 

1 

4 

3 

9 

Never 

0 

0 

0 

1 

0 

1 

Total 

1 

9 

22 

21 

6 

59 

(79) 
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Q42  Do  you  have  confidence  your  subordinates  understand  their 
responsibiiities  for  managing  risks?  *  Q46  Is  risk  management  given 
consideration  when  assessing  the  performance  of  subordinates? 


Value 

Approx.  Sig. 

Ordinal  by  Ordinal 

Kendall's  tau-b 

0.403 

0.000 

Nominal  by  Nominal 

Cramer’s  V 

0.304 

0.148 

(80) 


There  is  a  strong  positive  relationship  between  managers’  ratings  of  their  confidence  in 
subordinates’  understanding  of  the  limitations  on  their  authority  for  managing  risks  and  their  ratings 
of  how  consistently  risk  management  is  given  consideration  when  assessing  the  performance  of 
subordinates.  This  relationship  is  statistically  significant  at  99.9%  confidence. 


Q43  Do  you  have  confidence  your  subordinates  understand  the  limitations  on  their  authority  for 

managing  risks?  * 

Q46  Is  risk  management  given  consideration  when  assessing  the  performance  of 

subordinates? 

Is  risk  management  given  consideration  when 
assessing  the  performance  of  subordinates? 

Always 

Often 

Sometimes 

Rarely 

Never 

Total 

Do  you  have 

Always 

1 

1 

2 

1 

0 

5 

confidence  your 
subordinates 

Often 

0 

5 

12 

6 

0 

23 

understand  the 

Sometimes 

0 

1 

6 

8 

0 

15 

limitations  on  their 

Rarely 

0 

2 

2 

6 

3 

13 

authority  for 
managing  risks? 

Never 

0 

0 

0 

0 

3 

3 

Total 

1 

9 

22 

21 

6 

59 

(81) 


Q43  Do  you  have  confidence  your  subordinates  understand  the  limitations 
on  their  authority  for  managing  risks?  *  Q46  Is  risk  management  given 
consideration  when  assessing  the  performance  of  subordinates? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal  Kendall's  tau-b 

Nominal  by  Nominal  Cramer’s  V 

0.442 

0.465 

0.000 

0.000 

(82) 


Level  3  Managers  and  Level  4  Supervisors  and  Functional  Authorities 

Questions  57  through  69  asked  Level  3  and  Level  4  respondents  to  agree  or  disagree  with  a  variety 
of  statements.  The  responses  of  both  groups  are  presented  together  here  because  of  the  small  number 
of  Level  4  respondents. 

Many  of  the  responses  are  positive  (Strongly  Agree  and  Agree  =  over  80%).  For  example,  a  large 
majority  express  some  level  of  agreement  with  the  following  statements: 
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♦  Q57  I  can  disagree  with  my  supervisor  on  work-related  issues  without  fear  of  reprisal 
(80.0%); 

♦  Q58  1  never  withhold  risk  information  from  my  superior  (97.2%); 

♦  Q59  If  I  were  to  suggest  ways  to  improve  how  we  do  things,  my  supervisor  would  take 
them  seriously  (85.7%); 

♦  Q60  My  supervisor  creates  an  environment  where  it  is  acceptable  to  not  have  all  the 
answers  (82.9%); 

♦  Q63  In  my  work  unit,  we  learn  from  our  mistakes  and  do  what  it  takes  to  correct  them 
(85.7%);  and, 

♦  Q65  My  supervisor  trusts  me  to  make  good  decisions  in  high  risk/  emergency  situations 

(88.6%). 


Q57  1  can  disagree  with  my  supervisor  on  work-reiated  issues 
without  fear  of  reprisai 

Frequency 

Valid  Percent 

Strongly  Agree 

32 

45.7 

Agree 

24 

34.3 

Neither  Agree  nor  Disagree 

9 

12.9 

Disagree 

4 

5.7 

Strongly  Disagree 

1 

1.4 

Total 

70 

100.0 

Q58  1  never  withhoid  risk  information  from  my  superior 

Frequency 

Valid  Percent 

Strongly  Agree 

31 

44.3 

Agree 

37 

52.9 

Neither  Agree  nor  Disagree 

1 

1.4 

Disagree 

1 

1.4 

Total 

70 

100.0 

Q59  If  1  were  to  suggest  ways  to  improve  how  we  do  things,  my 
supervisor  wouid  take  them  seriousiy 

Frequency 

Valid  Percent 

Strongly  Agree 

27 

38.6 

Agree 

33 

47.1 

Neither  Agree  nor  Disagree 

8 

11.4 

Disagree 

2 

2.9 

Total 

70 

100.0 
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Q60  My  supervisor  creates  an  environment  where  it  is  acceptabie 
to  not  have  aii  the  answers 

Frequency 

Valid  Percent 

Strongly  Agree 

23 

32.9 

Agree 

35 

50.0 

Neither  Agree  nor  Disagree 

6 

8.6 

Disagree 

4 

5.7 

Strongly  Disagree 

2 

2.9 

Total 

70 

100.0 

Q63  in  my  work  unit,  we  ieam  from  our  mistakes  and  do  what  it 
takes  to  correct  them 

Frequency 

Valid  Percent 

Strongly  Agree 

20 

28.6 

Agree 

40 

57.1 

Neither  Agree  nor  Disagree 

5 

7.1 

Disagree 

4 

5.7 

Strongly  Disagree 

1 

1.4 

Total 

70 

100.0 

Q65  My  supervisor  trusts  me  to  make  good  decisions  in  high  risk/ 
emergency  situations 

Frequency 

Valid  Percent 

Strongly  Agree 

24 

34.3 

Agree 

38 

54.3 

Neither  Agree  nor  Disagree 

2 

2.9 

Disagree 

4 

5.7 

Strongly  Disagree 

2 

2.9 

Total 

70 

100.0 
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As  one  might  anticipate,  respondents  who  agree  that  their  superior  creates  an  environment  where  it 
is  acceptable  to  not  have  all  the  answers  are  also  more  likely  to  agree  that  they  can  disagree  with  a 
supervisor  on  work-related  issues  without  fear  of  reprisal.  A  Kendall’s  tau-b  test  statistic  indicates  a 
strong  positive  relationship  between  the  two  variables  which  is  significant  at  99.9%  confidence. 


Q60  My  supervisor  creates  an  environment  where  it  is  acceptabie  to  not  have  aii  the  answers  *  Q57 

1  can  disagree  with  my  supervisor  on  work-reiated  issues  without  fear  of  reprisai 

1  can  disagree  with  my  supervisor  on  work-related  issues 
without  fear  of  reprisal 

Strongly 

Agree 

Agree 

Neither 
Agree  nor 
Disagree 

Disagree 

Strongly 

Disagree 

Total 

My  supervisor 
creates  an 

Strongly 

Agree 

17 

5 

1 

0 

0 

23 

Agree 

13 

15 

5 

1 

1 

35 

environment 
where  it  is 

Neither 
Agree  nor 

1 

1 

3 

1 

0 

6 

acceptable  to  not 
have  all  the 

Disagree 

Disagree 

1 

1 

0 

2 

0 

4 

answers 

Strongly 

0 

2 

0 

0 

0 

2 

Total 

Disagree 

32 

24 

9 

4 

1 

70 

(89) 


Q60  My  supervisor  creates  an  environment  where  it  is  acceptabie  to 
not  have  aii  the  answers  *  Q57  i  can  disagree  with  my  supervisor  on 
work-reiated  issues  without  fear  of  reprisai 

Value 

Approx. 

Sig. 

Ordinal  by  Ordinal  Kendall's  tau-b 

Nominal  by  Nominal  Cramer's  V 

0.415 

0.370 

0.000 

0.001 
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Also  as  one  might  anticipate,  respondents  who  agree  that  their  superior  creates  an  environment 
where  it  is  acceptable  to  not  have  all  the  answers  are  also  more  likely  to  agree  that  if  they  were  to 
suggest  ways  to  improve  processes,  their  supervisor  would  take  them  seriously.  A  Kendall’s  tau-c 
test  statistic  indicates  a  moderate-strong  positive  relationship  between  the  two  variables  which  is 
significant  at  99.9%  confidence. 


Q60  My  supervisor  creates  an  environment  where  it  is  acceptabie  to  not  have  aii  the  answers  * 
Q59  If  I  were  to  suggest  ways  to  improve  how  we  do  things,  my  supervisor  would  take  them 

seriously 


If  I  were  to  suggest  ways  to  improve  how  we  do 
things,  my  supervisor  would  take  them  seriously 


Neither 

Strongly 

Agree  nor 

Agree 

Agree 

Disagree 

Disagree 

Total 

My  supervisor 

Strongly  Agree 

16 

7 

0 

0 

23 

creates  an 

Agree 

8 

22 

4 

1 

35 

environment  where  Neither  Agree 

0 

0 

it  is  acceptable  to  nor  Disagree 

0 

J 

not  have  all  the  Disagree 

2 

1 

0 

1 

4 

answers 

Strongly 

Disagree 

1 

0 

1 

0 

2 

Total 

27 

33 

8 

2 

70 

(91) 

Q60  My  supervisor  creates  an  environment  where  it  is  acceptable  to  not 
have  all  the  answers  *  Q59  If  I  were  to  suggest  ways  to  improve  how  we 

do  things,  my  supervisor  would  take  them  seriously 

Approx. 

Value 

Sig. 

Ordinal  by  Ordinal 

Kendall's  tau-c 

0.347 

0.000 

Nominal  by  Nominal 

Cramer's  V 

0.416 

0.000 

(92) 


However,  some  areas  of  concern  exist.  Only  half  of  respondents  agree  that  in  their  absence  a 
colleague  could  step  in  to  support  their  activities  (50.0%),  while  37. 1%  express  a  level  of 
disagreement  with  the  statement,  and  12.9%  neither  agree  nor  disagree. 


Q61  If  I  were  unavoidably  absent,  a  colleague  could  step  in  to 
support  my  activities  with  relative  ease 

Frequency 

Valid  Percent 

Strongly  Agree 

3 

4.3 

Agree 

32 

45.7 

Neither  Agree  nor  Disagree 

9 

12.9 

Disagree 

21 

30.0 

Strongly  Disagree 

5 

7.1 

Total 

70 

100.0 
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A  full  20%  of  respondents  neither  agree  nor  disagree  that  their  superior  encourages  innovation  as  an 
outcome  of  risk  taking. 


Q62  My  supervisor  encourages  innovation  as  an  outcome  of  risk 

taking 

Frequency 

Valid  Percent 

Strongly  Agree 

17 

24.3 

Agree 

35 

50.0 

Neither  Agree  nor  Disagree 

14 

20.0 

Disagree 

2 

2.9 

Strongly  Disagree 

2 

2.9 

Total 

70 

100.0 

Only  a  quarter  of  respondents  (27.2%)  express  some  level  of  agreement  with  the  statement  “My 
supervisor  makes  risk  management  a  priority  in  our  unit,”  while  the  majority  (52.9%)  neither  agree 
nor  disagree. 


Q64  My  supervisor  makes  risk  management  a  priority  in  our  unit 

Frequency 

Valid  Percent 

Strongly  Agree 

3 

4.3 

Agree 

16 

22.9 

Neither  Agree  nor  Disagree 

37 

52.9 

Disagree 

12 

17.1 

Strongly  Disagree 

2 

2.9 

Total 

70 

100.0 

Almost  a  quarter  of  respondents  (24.3%)  neither  agree  nor  disagree  that  it  is  important  to  follow 
rules  when  dealing  with  high  risk  or  emergency  situations,  while  nearly  15%  express  disagreement 
(14.3%). 


Q66  It  is  important  to  foiiow  ruies  when  deaiing  with  high  risk/ 
emergency  situations 

Frequency 

Valid  Percent 

Strongly  Agree 

11 

15.7 

Agree 

32 

45.7 

Neither  Agree  nor  Disagree 

17 

24.3 

Disagree 

9 

12.9 

Strongly  Disagree 

1 

1.4 

Total 

70 

100.0 

(96) 
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It  appears  that  respondents  who  agree  that  their  supervisor  trusts  them  to  make  good  decisions  in 
high  risk  or  emergency  situations  are  more  likely  to  agree  that  it  is  important  to  follow  rules  when 
dealing  with  high  risk  or  emergency  situations.  There  is  a  moderate  positive  relationship  between 
these  variables,  as  show  by  a  Kendall’s  tau-b  statistic,  which  is  significant  at  95%  confidence. 


Q65  My  supervisor  trusts  me  to  make  good  decisions  in  high  risk/  emergency  situations  *  Q66  it  is 
important  to  foiiow  ruies  when  deaiing  with  high  risk/  emergency  situations 

It  is  important  to  follow  rules  when  dealing  with  high  risk/ 
emergency  situations 

Strongly 

Agree 

Agree 

Neither 
Agree  nor 
Disagree 

Disagree 

Strongly 

Disagree 

Total 

My  supervisor 

Strongly  Agree 

6 

12 

4 

2 

0 

24 

trusts  me  to  make 

Agree 

4 

17 

10 

7 

0 

38 

good  decisions  in 
high  risk/ 

Neither  Agree 
nor  Disagree 

1 

1 

0 

0 

0 

2 

emergency 

Disagree 

0 

2 

2 

0 

0 

4 

situations 

Strongly 

0 

0 

1 

0 

1 

2 

Total 

Disagree 

11 

32 

17 

9 

1 

70 

(97) 


Q65  My  supervisor  trusts  me  to  make  good  decisions  in  high  risk/ 
emergency  situations  *  Q66  It  is  important  to  foiiow  ruies  when 
deaiing  with  high  risk/  emergency  situations 

Value 

Approx. 

Sig. 

Ordinal  by  Ordinal  Kendall's  tau-b 

Nominal  by  Nominal  Cramer's  V 

0.205 

0.399 

0.047 

0.000 

(98) 

There  appears  to  be  a  high  level  of  uncertainty  as  to  the  value  of  risk  management  training;  37.1%  of 
respondents  neither  agree  nor  disagree  with  the  statement  “Risk  management  training  is  a  good  use 
of  my  time.” 


Q67  Risk  management  training  is  a  good  use  of  my  time 

Frequency 

Valid  Percent 

Strongly  Agree 

7 

10.0 

Agree 

32 

45.7 

Neither  Agree  nor  Disagree 

26 

37.1 

Disagree 

2 

2.9 

Strongly  Disagree 

3 

4.3 

Total 

70 

100.0 
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While  a  high  percentage  of  respondents  express  some  level  of  disagreement  with  the  statement  “I 
sometimes  downplay  a  project's  risks  in  order  to  ensure  the  project  gets  approval,”  nearly  a  quarter 
neither  agree  nor  disagree,  while  another  1 0%  express  agreement.  This  is  an  interesting  observation 
when  compared  to  results  of  question  58,  where  nearly  100%  of  respondents  claim  to  never 
withhold  risk  information  from  their  superior. 


Q68  1  sometimes  downplay  a  project's  risks  in  order  to  ensure  the 
project  gets  approval 

Frequency 

Valid  Percent 

Agree 

7 

10.0 

Neither  Agree  nor  Disagree 

16 

22.9 

Disagree 

29 

41.4 

Strongly  Disagree 

18 

25.7 

Total 

70 

100.0 

(100) 

However,  when  compared,  the  relationship  between  the  variables  is  moderate-strongly  negative,  at 
99.9%  confidence.  This  means  that  managers  who  strongly  agree  that  they  would  never  withhold 
risk  information  are  also  likely  to  strongly  disagree  that  they  downplay  risk  information  to  ensure 
project  approval.  Overall,  there  is  no  significant  disparity  between  the  variables. 


Q58  I  never  withhold  risk  information  from  my  superior  *  Q68  I  sometimes  downplay  a  project's  risks 

in  order  to  ensure  the  project  gets  approval 


I  sometimes  downplay  a  project's  risks  in  order  to 
ensure  the  project  gets  approval 

Total 

Agree 

Neither 

Agree 

nor 

Disagree 

Disagree 

Strongly 

Disagree 

I  never  withhold  risk 

Strongly  Agree 

2 

2 

14 

13 

31 

information  from  my 

Agree 

4 

13 

15 

5 

37 

superior 

Neither  Agree  nor 

Disagree 

u 

\ 

u 

u 

\ 

Disagree 

1 

0 

0 

0 

1 

Total 

7 

16 

29 

18 

70 

(101) 


Q58  I  never  withhold  risk  information  from  my  superior*  Q68  I 
sometimes  downplay  a  project's  risks  in  order  to  ensure  the  project 

gets  approval 


Value 

Approx. 

Sig. 

Ordinal  by  Ordinal 

Kendall's  tau-b 

-0.398 

0.000 

Nominal  by  Nominal 

Cramer's  V 

0.340 

0.004 

(102) 
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Over  a  fifth  of  respondents  neither  agree  nor  disagree  that  knowing  how  peers  have  dealt  with  risk 
issues  is  valuable. 


Q69  I  find  it  vaiuabie  to  know  how  my  peers  have  deait  with  risk 
issues  in  their  work 

Frequency 

Valid  Percent 

Strongly  Agree 

13 

18.6 

Agree 

41 

58.6 

Neither  Agree  nor  Disagree 

15 

21.4 

Strongly  Disagree 

1 

1.4 

Total 

70 

100.0 

(103) 

It  appears  that  respondents  who  agree  that  risk  management  training  is  a  good  use  of  time  also  agree 
it  is  valuable  to  know  how  their  peers  have  dealt  with  risk  issues.  Statistical  testing  with  Kendall’s 
tau-c  indicates  a  moderate-strong  positive  relationship,  significant  at  99.9%  confidence. 


Q69  i  find  it  vaiuabie  to  know  how  my  peers  have  deait  with  risk  issues  in  their  work  *  Q67  Risk 

management  training  is  a  good  use  of  my  time 

Risk  management  training  is  a  good  use  of  my  time 

Neither 

Strongly 

Agree  nor 

Strongly 

Agree 

Agree 

Disagree 

Disagree 

Disagree 

Total 

I  find  it  valuable  to 

Strongly 

6 

3 

3 

0 

1 

13 

know  how  my  peers 

Agree 

have  dealt  with  risk 

Agree 

1 

27 

12 

1 

0 

41 

issues  in  their  work 

Neither  Agree 
nor  Disagree 

0 

2 

11 

1 

1 

15 

Strongly 

Disagree 

0 

0 

0 

0 

1 

1 

Total 

7 

32 

26 

2 

3 

70 

(104) 


Q69  I  find  it  vaiuabie  to  know  how  my  peers  have  deait  with  risk 
issues  in  their  work  *  Q67  Risk  management  training  is  a  good  use  of 

my  time 

Value 

Approx. 

Sig. 

Ordinal  by  Ordinal  Kendall's  tau-c 

Nominal  by  Nominal  Cramer's  V 

0.376 

0.543 

0.000 

0.000 

(105) 


Level  4  Supervisors  and  Functional  Authorities 

Due  to  the  low  number  of  Level  4  respondents  to  the  survey,  it  is  not  possible  to  draw  definitive 
conclusions  about  Level  4  impressions  of  Risk  Management  and  Accountability  within  the 
organization.  However,  in  some  cases,  these  limited  responses  may  indicate  areas  for  further 
investigation  among  a  more  comprehensive  sample  of  Level  4  functional  authorities. 
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Most  level  4  respondents  report  their  superior  is  inconsistent  in  setting  performance  objectives  (four 
of  seven),  and  one  reports  never  having  objectives  set. 


Q47  Does  your  superior  set  performance  objectives  for 
you? 

Frequency 

Valid  Percent 

Valid  Often 

2 

28.6 

Sometimes 

3 

42.9 

Rarely 

1 

14.3 

Never 

1 

14.3 

Total 

7 

100.0 

(106) 

Most  Level  4  respondents  report  that  performance  objectives  tend  to  lack  measurable  standards. 


Q48  Do  your  performance  objectives  inciude 
measurabie  standards? 

Frequency 

Valid  Percent 

Valid  Often 

1 

16.7 

Sometimes 

3 

50.0 

Rarely 

2 

33.3 

Total 

6 

100.0 

(107) 

Half  of  Level  4  respondents  (three  of  six)  indicate  their  performance  objectives  are  inconsistent  in 
their  support  of  superior’s  performance  objectives. 


Q49  Do  your  performance  objectives  support  your 
superior's  achievement  of  their  own  objectives? 

Frequency 

Valid  Percent 

Valid  Always 

1 

16.7 

Often 

1 

16.7 

Sometimes 

3 

50.0 

Uncertain/ 

Don't  know 

1 

16.7 

Total 

6 

100.0 

(108) 
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Four  of  six  Level  4  respondents  indicate  that  their  work  progress  is  only  sometimes  or  rarely 
compared  to  organizational  plans  such  as  milestones. 


Q50  Is  your  work  progress  compared  to  organizational 
plans  (i.e.  milestones  in  CPME)? 

Frequency 

Valid  Percent 

Valid  Often 

1 

16.7 

Sometimes 

2 

33.3 

Rarely 

2 

33.3 

Never 

1 

16.7 

Total 

6 

100.0 

(109) 

Level  4  respondents  report  that  their  performance  is  most  commonly  reviewed  against  general 
expectations. 


Q  51  How  does  your  superior  review  your  performance? 

Yes 

No 

against  general  expectations 

71.4% 

28.6% 

against  salary  admin  system  or  performance 
management  framework 

14.3% 

85.7% 

against  statement  of  duties 

42.9% 

57.1% 

against  agreed  performance  objectives 

28.6% 

71.4% 

against  my  PLP 

0.0% 

100.0% 

on  an  ad  hoc  basis 

14.3% 

85.7% 

other 

0.0% 

100.0% 

(110) 


Level  4  respondents  in  our  survey  appear  divided  in  their  understanding  of  their  roles, 
responsibilities  and  limits  on  their  authority  in  regards  to  risk  management. 


Q52  Do  you  understand  your  role  in  managing  risks? 

Frequency 

Valid  Percent 

Valid  Always 

1 

14.3 

Often 

2 

28.6 

Sometimes 

2 

28.6 

Rarely 

1 

14.3 

Never 

1 

14.3 

Total 

7 

100.0 

(Ill) 
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Q53  Do  you  understand  your  responsibilities  for 
managing  risks? 

Frequency 

Valid  Percent 

Valid  Always 

3 

42.9 

Often 

1 

14.3 

Sometimes 

3 

42.9 

Total 

7 

100.0 

(112) 


Q54  Do  you  understand  the  limitations  on  your  authority 
for  managing  risks? 

Frequency 

Valid  Percent 

Valid  Always 

2 

28.6 

Often 

2 

28.6 

Sometimes 

1 

14.3 

Rarely 

1 

14.3 

Never 

1 

14.3 

Total 

7 

100.0 

(113) 

Respondents  are  also  divided  in  the  frequency  with  which  they  are  consulted  for  risk  related  input 
on  projects  the  outcome  of  which  you  have  a  direct  or  related  stake. 


Q55  Are  you  consulted  for  risk  management  input  on 

projects  the  outcome  of  which  you  have  a  direct  or 

related  stake? 

Frequency 

Valid  Percent 

Valid  Always 

1 

16.7 

Often 

2 

33.3 

Sometimes 

2 

33.3 

Rarely 

1 

16.7 

Total 

6 

100.0 

(114) 
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The  majority  of  Level  4  respondents  distinguish  between  ongoing  duties  and  assigned  tasks. 


Q56  Do  you  see  a  distinction  between  ongoing  duties 
and  assigned  tasks? 

Frequency 

Valid  Percent 

Valid  Yes 

5 

71.4 

No 

1 

14.3 

Uncertain/ 

Don't  Know 

1 

14.3 

Total 

7 

100.0 

(115) 


General  Comments 

Themes  emerging  from  the  general  comments  (Q70)  section  of  our  survey  primarily  indicate  that 
respondents  see  weaknesses  in  the  area  of  risk  management  in  the  organization. 

“It  was  difficult  to  answer  some  of  the  questions  on  how  managers  evaluate  subordinates  with 
regards  to  risk  management.  For  Defence  Scientists,  it  is  not  one  of  the  seven  SAS 
characteristics.” 

“The  main  risk  to  program  delivery  continues  to  remain  procurement,  particularly  R&D 
contracting.” 

“The  authority  to  manage  risk  has  not  been  delegated  down.” 

“Currently,  our  organization  doesn't  want  any  risk  ...  we  don't  manage  risk  ...  instead,  we  avoid 
risk.” 

Many  managers  seem  unclear  as  to  who  has  responsibilities  for  risk  management;  some  believe  that 
risk  management  is  implicitly  understood  to  be  exclusive  to  the  job  of  Project  Manager. 

“Clear  roles  and  responsibilities  -with  commensurate  authority-  are  needed  for  Thrust  Leads, 
Thrust  Coordinators,  and  centre  Section  Heads.  This  is  often  the  source  of  tension  as  we 
develop  and  deliver  programs.” 

“PMs  are  responsible  for  risk  management  and  it  is  done  within  their  remit  as  PM.” 

“Tons  les  gestionnaires  font  de  la  gestion  de  risque,  de  faqon  implicite.  Ce  qui  nous  manque,  c'est 
une  methodologie  pour  le  faire  systematiquement  dans  toutes  les  initiatives  et  dans  tons  les 
projets  scientifiques,  organisationnels  ou  en  gestion  des  capacites.”'®* 

The  need  for  clearer  direction  on  Risk  Management  in  the  organization  also  extends  to  training: 

“Training  would  be  useful,  especially  seeing  examples  of  how  others  manage  risk,  and  what  the 
best  practices  are.” 


English  translation:  “All  managers  manage  risk  intuitively,  but  would  benefit  from  a  clear  methodology  to  support 
systematic  risk  management  of  all  activities.” 
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7.2  Pilot  Survey  3  Results 


Survey  invitations  were  sent  to  149  email  addresses;  of  those  2  addresses  were  no  longer  functional, 
resulting  in  147  total  invitations.  Of  the  147  survey  invitations  solicited,  there  were  77  responses,  of 
which  70  were  valid;  this  is  a  valid  response  rate  of  47.6%. 

Response  rates  by  management  level  were: 

L2:  6  valid  responses  of  1 1  possible  (54.5%  return); 

L3:  58  valid  responses  of  106  possible  (54.7%  return);  and, 

L4:  6  valid  responses  of  30  possible  (20%  return). 

Survey  responses  are  organized  here  by  subject  matter  for  the  following  reasons:  level  4  response 
rates  are  too  low  to  provide  meaningful  information  on  their  own;  most  questions  were  asked  of  two 
or  more  levels;  and  level  2  responses  need  to  be  reported  in  non-identifying  parameters. 

Only  Level  3  respondents  were  asked  to  identify  which  DRDC  centre  they  work  from.  For  the 
purposes  of  anonymity,  the  centres  have  been  categorized  into  two  groups:  NDHQ  Based  Centres 
(CORA,  DGMPRA,  Security  Sciences,  S&T  Operations,  Corporate  Services  and  Chief  of  Staff)  and 
Traditional  research  Centres  (Atlantic,  Valcartier,  Ottawa,  Toronto  and  Suffield). 


Q4  Type  of  Centre 

Frequency 

Valid 

Percent 

Traditional  Research  Centre 

43 

74.1 

NDHQ  Based  Centre 

15 

25.9 

Total 

58 

100 

(116) 


Budgets  and  Procurement 

Nearly  seventy  percent  (69.2%)  of  Level  3  respondents  say  their  superior  set  a  budget  tolerance  for 
last  year’s  fiscal  budget;  of  those,  83.3%  kept  expenditures  within  the  budget  tolerance. 


Q6  Did  your  superior  set  a  budget  tolerance  for  the  last  fiscal 

year? 

Frequency 

Valid  Percent 

Valid  Yes 

36 

69.2 

No 

16 

30.8 

Total 

52 

100.0 

(117) 
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Q6a  Were  your  expenditures  within  the  set  budget  toierance 
for  the  iast  fiscai  year? 

Frequency 

Valid  Percent 

Valid 

Yes 

30 

83.3 

No 

6 

16.7 

Total 

36 

100.0 

(118) 

A  little  over  half  of  Level  3  respondents  (53.8%)  say  their  unit  has  a  strategic  capital  plan  which 
reflects  the  unit’s  long  term  needs.  Nearly  two-thirds  (63.5%)  say  their  unit’s  capital  equipment  plan 
(specifically)  is  based  on  budget  allocation. 


Q7  Does  your  unit's  strategic  capitai  pian  refiect  iong  term 

needs? 

Frequency 

Valid  Percent 

Valid  Yes 

28 

53.8 

No 

24 

46.2 

Total 

52 

100.0 

(119) 


Q8  is  iocai  capitai  equipment  pian  based  on  budget 
aiiocation? 

Frequency 

Valid  Percent 

Valid  Yes 

33 

63.5 

No 

19 

36.5 

Total 

52 

100.0 

(120) 

Level  3  managers  who  agree  that  their  unit’s  strategic  capital  plan  reflects  long  term  needs  appear 
more  likely  to  also  agree  that  local  capital  equipment  plans  are  based  on  budget  allocation. 
Statistical  analysis  indicates  there  is  a  strong  positive  relationship  between  the  two  variables,  and 
that  the  relationship  is  statistically  significant  at  99.9%  confidence. 


Q7  Does  your  unit's  strategic  capitai  pian  refiect  iong  term  needs? 

pian  based  on  budget  aiiocation? 

*  Q8  Is  iocai  capitai  equipment 

Is  local  capital  equipment  plan  based 
on  budget  allocation? 

Total 

Yes 

No 

Does  your  unit's 

Yes 

23 

5 

28 

strategic  capital  plan 

reflect  long  term 

10 

14 

24 

needs? 

Total 

33 

19 

52 

(121) 
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Q7  Does  your  unit's  strategic  capitai  pian  refiect  iong  term  needs?  * 
Q8  is  iocai  capitai  equipment  pian  based  on  budget  aiiocation? 

Value 

df 

Asymp.  Sig. 
(2-sided) 

Phi 

0.419 

0.003 

Pearson  Chi-Square 

9.131 

1 

0.003 

(122) 

One  quarter  of  respondents  (25%)  report  that  the  number  of  procurement  specialists  in  their  unit  is 
‘Often’  or  ‘Always’  adequate  for  the  level  of  procurement  activity;  44.2%  say  the  number  is  ‘Never’ 
or  ‘Rarely’  adequate,  while  30.8%  say  it  is  ‘Sometimes’  adequate. 


Q10  is  the  number  of  procurement  speciaiists  in  your  centre 
adequate  for  the  ievei  of  procurement  activity? 

Frequency 

Valid  Percent 

Valid  Never 

4 

7.7 

Rarely 

19 

36.5 

Sometimes 

16 

30.8 

Often 

12 

23.1 

Always 

1 

1.9 

Total 

52 

100.0 

(123) 

Opinions  about  the  adequacy  of  the  number  of  procurement  specialists  appear  to  vary  by  centre, 
with  those  based  in  Traditional  Research  Centres  rating  the  adequacy  more  positively  than  those  in 
NDHQ  Based  Centres.  There  is  a  moderate-strong  relationship  between  these  two  variables  which  is 
significant  at  99%  confidence. 


Q4  Type  of  Centre  *  Q10  Is  the  number  of  procurement  speciaiists  in  your  centre  adequate  for 

the  ievei  of  procurement  activity? 

Q1 0  Is  the  number  of  procurement  specialists  in  your 
centre  adequate  for  the  level  of  procurement 
activity? 

Never 

Rarely 

Sometimes 

Often 

Always 

Total 

Type  of  Traditional  Research 
Centre  Centre 

1 

13 

11 

12 

1 

38 

NDHQ  Based  Centre 

3 

6 

5 

0 

0 

14 

Total 

4 

19 

16 

12 

1 

52 

(124) 


Type  of  Centre  *  Q10  Is  the  number  of  procurement  specialists  in  your 
centre  adequate  for  the  ievei  of  procurement  activity? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal  Kendall's  tau-c 

Nominal  by  Nominal  Cramer's  V 

-0.361 

0.435 

0.002 

0.043 

(125) 
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Level  3  managers  are  divided  as  to  the  ease  of  the  procurement  process.  53.8%  say  that  ‘None’  or 
‘Few’  of  the  internal  procurement  processes  are  easy  to  use,  while  46.2%  say  that  ‘Most’  or  ‘All’  are 
easy  to  use. 


Q11  How  many  internal  procurement  processes  are  easy  to 

use? 

Frequency 

Valid  Percent 

Valid  None 

5 

9.6 

Few 

23 

44.2 

Most 

21 

40.4 

All 

3 

5.8 

Total 

52 

100.0 

(126) 

Opinions  about  the  ease  of  use  of  internal  procurement  processes  appear  to  vary  by  centre,  with 
those  based  in  Traditional  Research  Centres  rating  more  of  the  processes  easy  to  use  than 
respondents  in  NDHQ  Based  Centres.  There  is  a  strong  relationship  between  these  two  variables 
which  is  significant  at  99%  confidence. 


Type  of  Centre  *  Q11  How  many  internal  procurement  processes  are  easy  to  use? 


Q1 1  How  many  internal  procurement 
processes  are  easy  to  use? 


None 

Few 

Most 

All 

Total 

Type  of 

Traditional  Research  Centre 

2 

13 

20 

3 

38 

Centre 

NDHQ  Based  Centre 

3 

10 

1 

0 

14 

Total 

5 

23 

21 

3 

52 

(127) 


Type  of  Centre  *  Q11  How  many  internal  procurement  processes  are 

easy  to  use? 

Value 

Approx.  Siq. 

Ordinal  by  Ordinal  Kendall's  tau-c 

Nominal  by  Nominal  Cramer's  V 

-0.453 

0.487 

0.000 

0.006 

(128) 
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Respondents  who  say  that  most  or  all  procurement  processes  tend  to  be  user-friendly  are  somewhat 
more  likely  to  feel  there  is  consistently  an  adequate  number  of  procurement  specialists  in  their  unit. 
A  Kendall’s  tau-c  statistical  test  shows  a  moderate-strong  positive  relationship  between  the  rankings 
of  the  two  variables  at  99.9%  confidence. 


Q11  How  many  internal  procurement  processes  are  easy  to  use?  *  Q10  Is  the  number  of  procurement 
specialists  in  your  centre  adequate  for  the  level  of  procurement  activity? 


Is  the  number  of  procurement  specialists  in  your  centre 
adequate  for  the  level  of  procurement  activity? 

Total 

Never 

Rarely 

Sometimes 

Often 

Always 

How  many  internal 

None 

0 

4 

1 

0 

0 

5 

procurement 

Few 

2 

12 

7 

2 

0 

23 

processes  are  easy  to 
use? 

Most 

2 

2 

7 

9 

1 

21 

All 

0 

1 

1 

1 

0 

3 

Total 

4 

19 

16 

12 

1 

52 

(129) 


Q11  How  many  internal  procurement  processes  are  easy  to  use?  *  Q10 

Is  the  number  of  procurement  specialists  in  your  centre  adequate  for 
the  level  of  procurement  activity? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal  Kendall's  tau-c 

0.340 

0.000 

Nominal  by  Nominal  Cramer's  V 

0.337 

0.125 

(130) 

Twenty-three  percent  (23%)  of  respondents  say  that  it  is  ‘Never’  or  ‘Rarely’  easy  to  determine  the 
progress  of  a  procurement  file,  42.3%  say  it  is  sometimes  easy  (and  sometimes  not),  while  34.6% 
say  it  is  either  ‘Often’  or  ‘Always’  easy  to  determine  the  progress  of  a  procurement  file. 


Q12  Can  clients  of  internal  procurement  processes  easily 
determine  the  progress  of  procurement  files? 

Frequency 

Valid  Percent 

Valid  Never 

2 

3.8 

Rarely 

10 

19.2 

Sometimes 

22 

42.3 

Often 

15 

28.8 

Always 

3 

5.8 

Total 

52 

100.0 

(131) 
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Opinions  about  the  ease  of  determining  the  progress  of  a  procurement  file  appear  to  vary  by  centre, 
with  those  based  in  Traditional  Research  Centres  rating  the  ease  of  determining  progress  more 
positively  than  those  in  NDHQ  Based  Centres.  There  is  a  strong  relationship  between  these  two 
variables  which  is  significant  at  99.9%  confidence. 


Q4  Type  of  Centre  *  Q12  Can  clients  of  internal  procurement  processes  easily  determine  the 

progress  of  procurement  files? 

Q12  Can  clients  of  internal  procurement  processes 
easily  determine  the  progress  of  procurement  files? 

Never 

Rarely 

Sometimes 

Often 

Always 

Total 

Type  of 

Traditional  Research 

1 

5 

14 

15 

3 

38 

Centre 

Centre 

NDHQ  Based  Centre 

1 

5 

8 

0 

0 

14 

Total 

2 

10 

22 

15 

3 

52 

(132) 


Type  of  Centre  Q12  Can  clients  of  internal  procurement  processes 
easily  determine  the  progress  of  procurement  files? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal  Kendall's  tau-c 

Nominal  by  Nominal  Cramer's  V 

-0.426 

0.457 

0.000 

0.028 

(133) 

There  appears  to  be  a  relationship  between  the  opinion  of  using  the  procurement  process  and  of 
determining  the  progress  of  the  procurement  request.  A  Kendall’s  tau-c  test  indicates  a  very  strong 
positive  relationship  between  the  two  variables,  which  is  statistically  significant  at  99.9% 
confidence. 


Q12  Can  clients  of  internal  procurement  processes  easily  determine  the  progress  of  procurement 
files?  *  Q11  How  many  internal  procurement  processes  are  easy  to  use? 

How  many  internal  procurement  processes  are 
easy  to  use? 

Total 

None 

Few 

Most 

All 

Can  clients  of  internal  Never 

2 

0 

0 

0 

2 

procurement  processes  Rarely 

2 

6 

2 

0 

10 

easily  determine  the  sometimes 

1 

14 

7 

0 

22 

progress  of  procurement 

files?  Often 

0 

3 

11 

1 

15 

Always 

0 

0 

1 

2 

3 

Total 

5 

23 

21 

3 

52 

Q12  Can  clients  of  internal  procurement  processes  easily  determine  the 
progress  of  procurement  files?  *  Q11  How  many  internal  procurement 
processes  are  easy  to  use? 


Value 

Approx.  Sig. 

Ordinal  by  Ordinal 

Kendall's  tau-c 

0.519 

0.000 

Nominal  by  Nominal 

Cramer's  V 

0.592 

0.000 

(134) 
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Most  respondents  report  having  a  positive  relationship  with  procurement  specialists.  Sixty-six 
percent  (66%)  classify  the  relationship  as  ‘Good’  or  ‘Excellent’,  and  30%  as  ‘Fair’;  only  4%  classify 
the  relationship  as  ‘Poor’. 


Q13  How  would  you  rate  the  quality  of  your  working 
relationship  with  your  centre's  procurement  specialists? 

Frequency 

Valid  Percent 

Valid  Poor 

2 

4.0 

Fair 

15 

30.0 

Good 

14 

28.0 

Excellent 

19 

38.0 

Total 

50 

100.0 

(136) 

Opinions  about  the  quality  of  the  working  relationship  with  procurement  specialists  appear  to  vary 
by  centre,  with  those  based  in  Traditional  Research  Centres  rating  the  quality  of  the  relationship 
more  positively  than  those  in  NDHQ  Based  Centres.  There  is  a  very  strong  relationship  between 
these  two  variables  which  is  significant  at  99.9%  confidence. 


Type  of  Centre  *  Q13  How  would  you  rate  the  quality  of  your  working  relationship  with  your  centre's 

procurement  specialists? 

Q1 3  How  would  you  rate  the  quality  of  your 
working  relationship  with  your  centre's 
procurement  specialists? 

Total 

Poor 

Fair 

Good 

Excellent 

Type  of  Traditional  Research  Centre 

0 

7 

13 

18 

38 

Centre  ndhq  Based  Centre 

2 

8 

1 

1 

12 

Total 

2 

15 

14 

19 

50 

(137) 


Type  of  Centre  *  Q13  How  would  you  rate  the  quality  of  your  working 
relationship  with  your  centre's  procurement  specialists? 

Value 

Approx.  Siq. 

Ordinal  by  Ordinal  Kendall's  tau-c 

Nominal  by  Nominal  Cramer's  V 

-0.504 

0.620 

0.000 

0.000 

(138) 
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The  more  positively  a  respondent  rated  the  procurement  relationship  the  more  likely  s/he  was  to  also 
rate  determining  the  progress  of  the  procurement  request  as  consistently  easy.  Kendall’s  tau-c  testing 
indicates  a  very  strong  positive  relationship  that  is  significant  at  99.9%  confidence. 


Q12  Can  clients  of  internal  procurement  processes  easily  determine  the  progress  of 
procurement  files?  *  Q13  How  would  you  rate  the  quality  of  your  working  relationship  with 

your  centre's  procurement  specialists? 

How  would  you  rate  the  quality  of  your  working 
relationship  with  your  centre's  procurement 
specialists? 

Total 

Poor 

Fair 

Good 

Excellent 

Can  clients  of  internal 

Never 

1 

1 

0 

0 

2 

procurement 

Rarely 

1 

5 

2 

1 

9 

processes  easily 
determine  the 

Sometimes 

0 

8 

9 

4 

21 

progress  of 

Often 

0 

1 

2 

12 

15 

procurement  files? 

Always 

0 

0 

1 

2 

3 

Total 

2 

15 

14 

19 

50 

(139) 


Q12  Can  clients  of  internal  procurement  processes  easily  determine  the 
progress  of  procurement  files?  *  Q13  How  would  you  rate  the  quality  of  your 
working  relationship  with  your  centre's  procurement  specialists? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal  Kendall's  tau-c 

Nominal  by  Nominal  Cramer's  V 

0.530 

0.484 

0.000 

0.000 

(140) 

Additionally,  the  more  positively  a  respondent  rated  the  procurement  relationship  the  more  likely 
s/he  was  to  also  rate  the  procurement  process  as  consistently  user-friendly.  Kendall’s  tau-c  testing 
indicates  a  very  strong  positive  relationship  that  is  significant  at  99.9%  confidence. 


Q11  How  many  internal  procurement  processes  are  easy  to  use?  *  Q13  How  would  you  rate  the 
quality  of  your  working  relationship  with  your  centre's  procurement  specialists? 


How  would  you  rate  the  quality  of  your  working 
relationship  with  your  centre's  procurement  specialists? 

Poor 

Fair 

Good 

Excellent 

Total 

How  many  internal 

None 

1 

4 

0 

0 

5 

procurement 

Few 

1 

10 

7 

4 

22 

processes  are  easy  to 
use? 

Most 

0 

1 

7 

12 

20 

All 

0 

0 

0 

3 

3 

Total 

2 

15 

14 

19 

50 

(141) 
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Q11  How  many  internal  procurement  processes  are  easy  to  use?  *  Q13  How 
would  you  rate  the  quality  of  your  working  relationship  with  your  centre's 
procurement  specialists? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal 

Kendall's  tau-c 

0.539 

0.000 

Nominal  by  Nominal 

Cramer's  V 

0.429 

0.001 

(142) 

Most  respondents  report  having  a  positive  relationship  with  their  unit’s  financial  specialists.  Eighty 
percent  (80.7%)  classify  the  relationship  as  ‘Good’  or  ‘Excellent’,  and  15.4%  as  ‘Fair’;  only  3.8% 
classify  the  relationship  as  ‘Poor’. 


Q9  How  would  you  rate  the  quality  of  your  working 
relationship  with  your  centre's  financial  specialists? 

Frequency 

Valid  Percent 

Valid  Poor 

2 

3.8 

Fair 

8 

15.4 

Good 

19 

36.5 

Excellent 

23 

44.2 

Total 

52 

100.0 

(143) 

There  is  no  statistically  significant  relationship  between  how  a  respondent  rated  his/her  relationship 
with  procurement  specialists  versus  the  quality  of  their  relationship  with  financial  specialists. 


Q9  How  would  you  rate  the  quality  of  your  working  relationship  with  your  centre's  financial 
specialists?  *  Q13  How  would  you  rate  the  quality  of  your  working  relationship  with  your 

centre's  procurement  specialists? 

How  would  you  rate  the  quality  of  your  working 
relationship  with  your  centre's  procurement 
specialists? 

Poor 

Fair 

Good 

Excellent 

Total 

How  would  you  rate 

Poor 

0 

0 

0 

2 

2 

the  quality  of  your 

Fair 

1 

4 

1 

2 

8 

working  relationship 

Good 

1 

6 

9 

2 

18 

with  your  centre's 
financial  specialists? 

Excellent 

0 

5 

4 

13 

22 

Total 

2 

15 

14 

19 

50 

(144) 


152 


DRDC  Atlantic  CR  2009-135 


Q9  How  would  you  rate  the  quality  of  your  working  relationship  with  your 
centre's  financial  specialists?  *  Q13  How  would  you  rate  the  quality  of 
your  working  relationship  with  your  centre's  procurement  specialists? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal  Kendall's  tau-b 

Nominal  by  Nominal  Cramer's  V 

0.259 

0.348 

0.055 

0.034 

(145) 


Auditing  and  Inventory 

Just  over  half  of  respondents  (55.2%)  agree  that  an  auditor  would  get  sufficient  information  using 
files  only  during  an  audit.  Of  those  who  disagree,  61.5%  say  that  the  auditor  would  need  to  consult 
with  a  manager  to  a  moderate  degree  to  get  sufficient  information  to  complete  an  audit. 


Q14  Would  an  auditor  get  a  complete  picture  of  activities  in  your  unit 
using  only  files  and  other  records? 

Frequency  Valid  Percent 

Valid  Yes 

No 

Total 

32  55.2 

26  44.8 

58  100.0 

(146) 


Q15  To  what  degree  would  an  auditor  need  to  consult  with  you  or  a 
subordinate  to  get  a  complete  picture  of  activities  in  your  unit? 

Frequency 

Valid  Percent 

Valid  Significant 

4 

15.4 

Moderate 

16 

61.5 

Minor 

6 

23.1 

Total 

26 

100.0 

(147) 
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The  majority  of  respondents  (74.4%)  responsible  for  controlling  an  inventory  of  accountable  items 
have  surveyed  those  items  within  the  last  year. 


Q16  When  did  you  ora  subordinate  iast  survey  the  inventory  for  which 
you  are  accountabie? 

Frequency 

Valid  Percent 

Valid  Never 

1 

2.6 

Within  the  last  6 
months 

14 

35.9 

Within  the  last  7  to  12 
months 

15 

38.5 

Within  the  last  13  to 

24  months 

5 

12.8 

More  than  24  months 
ago 

4 

10.3 

Total 

39 

100.0 

(148) 


Files  and  Documents 

Personnel  files  appear  to  be  accessible  to  employees  outside  of  HR  staff  Thirty-seven  percent 
(37.1%)  of  respondents  reported  that  access  to  personnel  files  was  not  restricted  to  HR  staff,  while 
another  22.9%  did  not  know  if  access  was  restricted  or  not. 


Q56  in  your  centre,  is  access  to  personnei  fiies 
restricted  to  HR  staff? 

Frequency 

Valid  Percent 

Valid  Yes 

28 

40.0 

No 

26 

37.1 

Don't 

Know 

16 

22.9 

Total 

70 

100.0 

(149) 


There  is  no  statistically  significant  relationship  between  how  respondents  answered  and  the  centre 
from  which  they  work;  the  following  table  is  for  information  purposes  only. 


Q4  Type  of  Centre  *  Q56  In  your  centre,  is  access  to  personnei  fiies  restricted  to  HR  staff? 

Q56  In  your  centre,  is  access  to 
personnel  files  restricted  to  HR  staff? 

Yes 

No 

Don't  Know 

Total 

Type  of  Centre  Traditional  Research 

Centre 

18 

15 

10 

43 

NDHQ  Based  Centre 

5 

8 

2 

15 

Total 

23 

23 

12 

58 

(150) 
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All  respondents  (100%)  report  that  classified  correspondence  is  filed  securely  (Q58),  but  a  quarter  of 
respondents  (25.7%)  do  not  know  if  classified  correspondence  is  tracked. 


Q57  In  your  centre,  is  classified  correspondence 
tracked? 

Frequency 

Valid  Percent 

Valid  Yes 

50 

71.4 

No 

2 

2.9 

Don't 

Know 

18 

25.7 

Total 

70 

100.0 

(151) 


Equipment  maintenance  and  repair 

Responsibility  for  routine  maintenance  appears  to  be  more  likely  to  be  formally  assigned  in  the  case 
of  program  related  equipment  (62.5%)  than  for  infrastructure  related  equipment  (52.0%). 


Q47  Is  routine  maintenance  of  program-related  equipment  formally 
assigned  to  a  designated  individual  in  your  unit? 

Frequency 

Valid  Percent 

Valid  Yes 

20 

62.5 

No 

12 

37.5 

Total 

32 

100.0 

(152) 


Q48  Is  routine  maintenance  of  infrastructure-related  equipment  formally 
assigned  to  a  designated  individual  in  your  unit? 

Frequency 

Valid  Percent 

Valid 

Yes 

26 

52.0 

No 

24 

48.0 

Total 

50 

100.0 

(153) 
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It  appears  that  units  which  have  assigned  responsibility  for  routine  maintenance  of  one  type  of 
equipment  are  more  likely  to  assign  responsibility  for  the  other.  Fisher’s  Exact  Test  and  Phi  show  a 
very  strong  relationship  between  these  variables,  which  is  statistically  significant  at  99%  confidence. 


Q47  Is  routine  maintenance  of  program-reiated  equipment  formaiiy  assigned  to  a 
designated  individuai  in  your  unit?  *  Q48  Is  routine  maintenance  of  infrastructure- 
reiated  equipment  formaiiy  assigned  to  a  designated  individuai  in  your  unit? 

Is  routine  maintenance  of 
infrastructure-related  equipment 
formally  assigned  to  a  designated 
individual  in  your  unit? 

Yes 

No 

Total 

Is  routine  maintenance  of 
program-related  equipment 

Yes 

16 

4 

20 

formally  assigned  to  a 
designated  individual  in  your 
unit? 

No 

3 

9 

12 

Total 

19 

13 

32 

(154) 


Q47  Is  routine  maintenance  of  program-reiated  equipment  formaiiy  assigned  to  a 
designated  individuai  in  your  unit?  *  Q48  is  routine  maintenance  of  infrastructure- 
reiated  equipment  formaiiy  assigned  to  a  designated  individuai  in  your  unit? 

Value 

Asymp.  Sig. 
(2-sided) 

Exact  Sig. 

(1 -sided) 

Approx.  Sig. 

Pearson  Chi-Square 

Phi 

Fisher's  Exact  Test 

9.406(b) 

0.542 

0.002 

0.003 

0.002 

b.  1  cells  (25.0%)  have  expected  count  less  than  5.  The  minimum  expected  count  is  4.88. 

(155) 

The  majority  of  respondents  consistently  find  the  process  for  requesting  special  maintenance  or 
repair  of  equipment  easy  to  use  (62.6%). 


Q49  When  equipment  needs  speciai  maintenance  or  repair,  is  the  process 
for  the  request  easy  to  use? 

Frequency 

Valid  Percent 

Valid 

Never 

1 

1.6 

Rarely 

4 

6.3 

Sometimes 

19 

29.7 

Often 

28 

43.8 

Always 

12 

18.8 

Total 

64 

100.0 

(156) 
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Respondents  are  divided  as  to  the  timeliness  of  repair  requests  being  addressed;  51.6%  report  that 
the  repairs  are  ‘Rarely’  or  only  ‘Sometimes’  addressed  in  a  timely  and  effective  manner,  while  the 
remaining  48.4%  report  that  repairs  are  ‘Often’  or  ‘Always’  addressed  in  a  timely  and  effective 
manner. 


Q50  Are  equipment  repairs  addressed  in  a  timely  and  effective  manner? 

Frequency 

Valid  Percent 

Valid  Rarely 

4 

6.3 

Sometimes 

29 

45.3 

Often 

26 

40.6 

Always 

5 

7.8 

Total 

64 

100.0 

(157) 

There  appears  to  be  a  relationship  between  a  respondent’s  assessment  of  the  ease  of  use  of  the  repair 
request  process  and  the  assessment  of  the  timeliness  and  effectiveness  of  the  repair.  The  relationship 
between  the  ranking  of  the  two  variables  is  strong  and  positive,  with  statistical  significance  at  99.9% 
confidence. 


Q49  When  equipment  needs  special  maintenance  or  repair,  is  the  process  for  the  request 

easy  to  use?  * 

Q50  Are  equipment  repairs  addressed  in  a  timely  and  effective  manner? 

Are  equipment  repairs  addressed  in  a  timely  and 

effective  manner? 

Rarely 

Sometimes 

Often 

Always 

Total 

When  equipment 
needs  special 

Never 

1 

0 

0 

0 

1 

maintenance  or 

Rarely 

2 

2 

0 

0 

4 

repair,  is  the 
process  for  the 

Sometimes 

1 

13 

5 

0 

19 

request  easy  to 

Often 

0 

9 

18 

1 

28 

use? 

Always 

0 

5 

3 

4 

12 

Total 

4 

29 

26 

5 

64 

(158) 


Q49  When  equipment  needs  special  maintenance  or  repair,  is  the  process 
for  the  request  easy  to  use?  *  Q50  Are  equipment  repairs  addressed  in  a 
timely  and  effective  manner? 


Value 

Approx.  Sig. 

Ordinal  by  Ordinal 

Kendall's  tau-b 

0.455 

0.000 

Nominal  by  Nominal 

Cramer's  V 

0.524 

0.000 

(159) 


There  are  no  relationships  between  having  an  individual  designated  as  responsible  for  routine 
maintenance  of  equipment  and  either  the  ease  of  use  or  the  timeliness  of  the  repair  processes. 
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Project  Management 


All  respondents  were  asked  to  rate  how  frequently  five  broad  goals  are  explicitly  a  factor  in  the 
work  of  their  unit.  These  goals  are  based  on  alignment  with  the  Canada  S&T  Strategy.  The  two  most 
consistent  goals  in  a  unit’s  work  were  ‘Expansion  of  the  body  of  knowledge’  (71.5%)  and  ‘Mission 
directed  needs  for  R&D  results’  (70.0%). 


Q55  How  frequently  are  the  following  broad  goals  an  explicit  factor  in  your  unit's  work? 

Never (%) 

Inconsistently  (%) 

Consistently  (%) 

Expansion  of  the  body  of 
knowledge 

2.9 

25.7 

71.5 

Mission  directed  needs  for 

R&D  results 

12.9 

17.1 

70.0 

Workforce  development 

1.4 

41.5 

57.1 

Maintenance  of  national  S&T 
infrastructure  and  capacity 

14.3 

38.6 

47.1 

Commercialization 

21.4 

52.9 

25.7 

(160) 


There  are  moderate  to  strong,  positive  relationships  among  all  the  broad  goal  factors,  and  all  are 
statistically  significant  with  at  least  95%  confidence,  with  the  following  exceptions:  there  is  no 
statistically  significant  relationship  between  how  often  a  unit’s  work  involves  ‘workforce 
development’  and  ‘commercialization’;  and  there  is  no  statistically  significant  relationship  between 
how  often  a  unit’s  work  involves  ‘workforce  development’  and  ‘mission  directed  needs  for  R&D 
results’. 

More  than  a  third  (39.6%)  of  projects  managed  within  a  given  unit  do  not  utilize  a  responsibility 
matrix  or  equivalent  to  formally  assign  responsibilities. 


Q18  Does  Project  Management  documentation 
formally  assign  responsibilities? 

Frequency 

Valid  Percent 

Valid 

Yes 

29 

60.4 

No 

19 

39.6 

Total 

48 

100 

(161) 

Two-thirds  of  respondents  say  that  either  Resources  (64.6%)  or  Schedules  (64.6%)  are  likely  to 
play  a  role  in  the  failure  of  a  project,  one  third  (3 1 .3%)  report  that  Goals  are  likely  to  play  a  role  in 
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project  failures,  while  one-quarter  (24.9%)  say  the  Quality  of  Project  Milestones  are  likely  to  play  a 
role  in  project  failures. 


Q19  If  a  project  managed  within  your  unit  faiis,  how  iikeiy 
are  the  foiiowing  to  have  piayed  a  roie  in  that  faiiure? 

Very/ 

Somewhat 

Unlikely 

Neutral 

Very/ 

Somewhat 

Likely 

Resources 

18.8% 

16.7% 

64.6% 

Schedule 

14.6% 

20.8% 

64.6% 

Goals 

45.8% 

22.9% 

31.3% 

Milestone 

Quality 

43.8% 

31.3% 

24.9% 

(162) 

Respondents  report  making  project  results  available  through  a  number  of  methods,  the  most  popular 
being  Presentations  (77.6%)  and  Meetings  (77.6%). 


Q20  How  do  you  make  interim  orfinai 
resuits  avaiiabie  to  stakehoiders,  ciients 
or  partners? 

Presentations 

77.6 

Meetings 

77.6 

Standard  reports 

56.9 

Briefing  Notes 

48.3 

Demonstrations 

46.6 

Conference  Papers 

29.3 

Journals 

27.6 

Other 

15.5 

(163) 


Human  Resource  Management 

Eighty-five  percent  (85.0%)  of  respondents  report  that  their  subordinates’  Personal  Learning  Plans 
(PLPs)  are  linked  to  the  appropriate  Knowledge,  Skills,  and  Abilities  (KSA)  required  for  current  and 
future  roles. 


Q24  Are  subordinates'  PLPs  iinked  to  appropriate  KSA  required  for  current 

and  future  roies? 

Frequency 

Valid  Percent 

Valid  Sometimes 

9 

15.0 

Often 

35 

58.3 

Always 

16 

26.7 

Total 

60 

100.0 

(164) 
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Three-quarters  (78.3%)  of  respondents  have  discussed  methods  of  passing  on  corporate  knowledge 
with  subordinates. 


Q25  Have  you  discussed  with  subordinates  various  means  of  passing  on 

corporate  knowiedge? 

Frequency 

Valid  Percent 

Valid  Yes 

47 

78.3 

No 

13 

21.7 

Total 

60 

100.0 

(165) 

Eighty  percent  (80%)  of  respondents  report  consistently  encouraging  subordinates  to  work  in  teams 
and  cross  train  when  appropriate. 


Q26  Do  you  encourage  subordinates  to  work  in  teams  and  cross-train? 

Frequency 

Valid  Percent 

Valid  Rarely 

2 

3.3 

Sometimes 

10 

16.7 

Often 

25 

41.7 

Always 

23 

38.3 

Total 

60 

100.0 

(166) 

The  majority  (88.3%)  of  respondents  have  considered  bringing  in  a  new  employee  while  the 
incumbent  remains  in  order  to  facilitate  knowledge  transfer,  while  69.8%  have  actually  done  so. 


Q27  Have  you  considered  options  to  bring  in  a  new  empioyee  whiie  to 
incumbent  remains  in  order  to  faciiitate  knowiedge  transfer? 

Frequency 

Valid  Percent 

Valid  Yes 

53 

88.3 

No 

7 

11.7 

Total 

60 

100.0 

(167) 


Q27a  Have  you  ever  brought  in  a  new  empioyee  whiie  to  incumbent 
remained  in  order  to  faciiitate  knowiedge  transfer? 

Frequency 

Valid  Percent 

Valid 

Yes 

37 

69.8 

No 

16 

30.2 

Total 

53 

100.0 

(168) 


160 


DRDC  Atlantic  CR  2009-135 


More  than  three-quarters  of  respondents  (80%)  report  that  their  unit  does  not  have  sufficient  FTE  to 
fulfill  its  mandate. 


Q28  Does  your  unit  have  sufficient  FTEs  to  successfui  fuifiii  your  unit's 

mandate? 

Frequency 

Valid  Percent 

Valid  Yes 

12 

20.0 

No 

48 

80.0 

Total 

60 

100.0 

(169) 

Many  respondents  (66.7%)  report  that  key  positions  are  only  ‘Rarely’  or  ‘Sometimes’  filled  quickly. 


Q29  Are  key  positions  fiiied  quickiy? 

Frequency 

Valid  Percent 

Valid 

Never 

1 

1.7 

Rarely 

13 

21.7 

Sometimes 

27 

45.0 

Often 

18 

30.0 

Always 

1 

1.7 

Total 

60 

100.0 

(170) 

Eighty  percent  (80%)  of  respondents  report  that  new  employees  in  key  positions  ‘Often’  or 
‘Always’  perform  effectively  within  a  reasonable  timeframe. 


Q30  Do  new  employees  in  key  positions  perform  effectively  in  their  role 
within  a  reasonable  timeframe? 

Frequency 

Valid  Percent 

Valid  Sometimes 

12 

20.0 

Often 

45 

75.0 

Always 

3 

5.0 

Total 

60 

100.0 

(171) 
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Over  eighty  percent  (83.3%)  of  respondents  say  that  there  are  employees  in  their  unit  who  are  ready 
to  compete  for  key  positions  and  areas. 


Q31  Are  there  qualified  employees  in  your  unit  ready  to  compete  for  key 

positions  and  areas? 

Frequency 

Valid  Percent 

Valid  Yes 

50 

83.3 

No 

10 

16.7 

Total 

60 

100.0 

(172) 

Almost  all  respondents  (91.7%)  say  they  identify  ‘high  performing  employees  for  succession 
planning,  despite  not  having  an  official  DRDC  succession  plan. 


Q32  Do  you  identify  'high-performing'  employees  for  succession 

planning? 

Frequency 

Valid  Percent 

Valid  Yes 

55 

91.7 

No 

5 

8.3 

Total 

60 

100.0 

(173) 

There  are  no  statistically  significant  relationships  between  how  quickly  key  positions  are  filled 
(Q29)  and  the  identification  of  high  performing  employees  (Q32),  nor  how  effectively  employees  in 
new  roles  perform  (Q30).  There  is  also  no  statistically  significant  relationship  between  the 
identification  of  high  performing  employees  (Q32)  and  how  effectively  employees  in  new  roles 
perform  (Q30). 

Many  respondents  (68.8%)  report  that  it  is  ‘Very  Important’  to  their  job  performance  to  stay  abreast 
of  new  ideas,  methods,  and  technology  in  their  field. 


Q51  How  important  is  it  to  your  job  performance  to  stay  abreast  of  new 
ideas,  methods,  and  technology  in  your  field? 

Frequency 

Valid  Percent 

Valid  Very  Unimportant 

3 

4.7 

Somewhat 

Unimportant 

1 

1.6 

Somewhat  Important 

16 

25.0 

Very  Important 

44 

68.8 

Total 

64 

100.0 

(174) 
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Sixty-one  percent  (61.0%)  of  respondents  expressed  some  level  of  satisfaction  with  the  resources 
and/or  opportunities  provided  by  DRDC  to  stay  abreast  of  new  ideas,  methods  and  technology, 
while  the  remaining  thirty-nine  percent  were  divided  between  some  level  of  dissatisfaction  and 
neutrality. 


Q52  Are  you  satisfied  with  the  resources  and/or  opportunities  provided  by 
DRDC  to  stay  abreast  of  new  ideas,  methods  and  technoiogy  in  your  fieid? 

Frequency 

Valid  Percent 

Valid  Very  Dissatisfied 

3 

4.7 

Somewhat 

Dissatisfied 

10 

15.6 

Neither  Satisfied  nor 

Dissatisfied 

12 

18.8 

Somewhat  Satisfied 

28 

43.8 

Very  Satisfied 

11 

17.2 

Total 

64 

100.0 

(175) 

There  is  no  relationship  between  the  level  of  importance  a  respondent  ascribes  to  staying  abreast  of 
new  ideas  in  their  field  and  their  level  of  satisfaction  with  resources  to  do  so  at  DRDC.  The  table 
below  is  provided  for  information  purposes  only. 


Q51  How  important  is  it  to  your  job  performance  to  stay  abreast  of  new  ideas,  methods,  and 
technoiogy  in  your  fieid?  *  Q52  Are  you  satisfied  with  the  resources  and/or  opportunities  provided  by 
DRDC  to  stay  abreast  of  new  ideas,  methods  and  technoiogy  in  your  fieid? 

Q52  Are  you  satisfied  with  the  resources  and/or  opportunities 
provided  by  DRDC  to  stay  abreast  of  new  ideas,  methods  and 
technology  in  your  field? 

Very 

Dissatisfied 

Somewhat 

Dissatisfied 

Neither 
Satisfied  nor 
Dissatisfied 

Somewhat 

Satisfied 

Very 

Satisfied 

Total 

Q51  How  important 

Very 

Unimportant 

0 

0 

0 

2 

1 

3 

is  it  to  your  job 
performance  to 
stay  abreast  of  new 

Somewhat 

Unimportant 

0 

0 

1 

0 

0 

1 

ideas,  methods, 
and  technology  in 

Somewhat 

Important 

1 

0 

6 

7 

2 

16 

your  field? 

Very 

Important 

2 

10 

5 

19 

8 

44 

Total 

3 

10 

12 

28 

11 

64 

(176) 
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Nearly  three-quarter  of  respondents  (70.3%)  have  worked  with  students  as  a  part  of  their  unit  in  the 
past.  Of  these,  40%  characterized  the  experience  as  ‘Excellent’,  and  another  53.3%  as  ‘Good’. 


Q53  Have  you  worked  with  students  in  your  unit  in  the  past? 

Frequency 

Valid  Percent 

Valid  Yes 

45 

70.3 

No 

19 

29.7 

Total 

64 

100.0 

(177) 


Q54  How  wouid  you  rate  your  unit's  experience  working  with 

students? 

Frequency 

Valid  Percent 

Valid  Fair 

3 

6.7 

Good 

24 

53.3 

Excellent 

18 

40.0 

Total 

45 

100.0 

(178) 


Employment  Equity 

One  third  of  Level  2  respondents  believe  designated  Employment  Equity  groups  are  adequately 
represented  in  feeder  groups  for  key  positions,  and  marginally  more  (50%)  say  their  centre  offers 
Professional  Development  and  training  opportunities  specifically  to  identify  and  develop  talent 
among  members  of  designated  EE  groups. 


Q2  Are  designated  EE  groups  adequateiy 
represented  in  feeder  groups  for  key 
positions? 

Frequency 

Valid 

Percent 

Valid  Yes 

2 

33.3 

No 

4 

66.7 

Total 

6 

100.0 

(179) 


Q3  Does  your  centre  offer  PD  and  training 
to  identify  and  deveiop  taient  among  EE 
groups? 

Frequency 

Valid 

Percent 

Valid  Yes 

3 

50.0 

No 

3 

50.0 

Total 

6 

100.0 

(180) 
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A  Fisher’s  Exact  Test  indicates  no  statistically  significant  relationship  between  the  two  variables. 


Q2  Are  designated  EE  groups  adequately  represented  in  feeder 
groups  for  key  positions?  *  Q3  Does  your  centre  offer  PD  and  training 
to  identify  and  develop  talent  among  EE  groups? 

Exact  Sig.  (2-sided) 

Exact  Sig.  (1 -sided) 

Fisher's  Exact  Test 

0.400 

0.200 

(181) 


S&T  Peer  Reviews 

Almost  half  of  level  three  respondents  (46.9%)  came  from  units  which  are  subject  to  S&T  peer 
review  process.  Of  those  which  are  subject  to  Peer  review,  most  (46.2%)  have  had  a  review  within 
the  last  year;  a  significant  number  (19.2%)  either  had  never  had  a  review  or  had  gone  more  than  5 
years  without  one. 


Q33  Is  your  unit  subject  to  S&T  peer  reviews? 

Freguency 

Valid  Percent 

Valid 

Yes 

30 

46.9 

No 

26 

40.6 

Don't  Know 

8 

12.5 

Total 

64 

100.0 

(182) 


Q34  When  was  the  last  peer  review  of  an  S&T  activity  in  your  unit? 

Freguency 

Valid  Percent 

Valid  Never 

1 

3.8 

One  year  ago 

12 

46.2 

Two  years  ago 

2 

7.7 

Three  years  ago 

3 

11.5 

Four  years  ago 

3 

11.5 

Five  years  ago 

1 

3.8 

More  than  five  years 
ago 

4 

15.4 

Total 

26 

100.0 

(183) 
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Most  respondents  (91.7%)  reported  that  the  outcome  of  their  unit’s  last  peer  review  was  either 
somewhat  or  very  positive.  No  one  reported  negative  outcomes. 


Q35  Overall,  what  was  the  outcome  of  your  last  peer  review? 

Frequency 

Valid  Percent 

Valid  Neutral 

2 

8.3 

Somewhat  Positive 

9 

37.5 

Very  Positive 

13 

54.2 

Total 

24 

100.0 

(184) 

Many  respondents  (70.8%)  found  the  feedback  from  their  last  peer  review  to  be  valuable. 


Q36  Overall,  did  your  last  peer  review  offer  valuable  feedback? 

Frequency 

Valid  Percent 

Valid  Not  Valuable 

2 

8.3 

Neutral 

5 

20.8 

Valuable 

17 

70.8 

Total 

24 

100.0 

(185) 

Seventy  percent  (70.8%)  say  that  the  feedback  they  received  had  ‘Some  Impact’  on  their  unit’s 
capabilities  and  competencies,  and  66.7%  characterized  that  impact  as  positive. 


Q37  In  general,  how  much  impact  does  peer  review  have  on  your  unit's 
capabilities  and  competencies? 

Frequency 

Valid  Percent 

Valid  No  Impact 

4 

16.7 

Some  Impact 

17 

70.8 

Significant  Impact 

3 

12.5 

Total 

24 

100.0 

(186) 
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Q38  In  general,  what  kind  of  impact  does  peer  review  have  on  your  unit's 
capabilities  and  competencies? 

Frequency 

Valid  Percent 

Valid  Neutral/  No  Impact 

8 

33.3 

Positive  Impact 

16 

66.7 

Total 

24 

100.0 

(187) 

Sixty-six  percent  (66.7%)  say  that  the  feedback  they  received  had  ‘Some  Impact’  on  their  unit’s 
project  outcomes,  but  58.3%  characterized  that  impact  as  neutral  or  non-existent. 


Q39  In  general,  how  much  impact  does  peer  review  have  on  the  outcomes 

of  your  projects? 

Frequency 

Valid  Percent 

Valid 

No  Impact 

7 

29.2 

Some  Impact 

16 

66.7 

Significant  Impact 

1 

4.2 

Total 

24 

100.0 

(188) 


Q40  In  general,  what  kind  of  impact  does  peer  review  have  on  the  outcomes 

of  your  projects? 

Frequency 

Valid  Percent 

Valid  Neutral/  No  Impact 

14 

58.3 

Positive  Impact 

10 

41.7 

Total 

24 

100.0 

(189) 
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It  appears  that  the  more  recently  an  S&T  peer  review  occurred  the  more  impact  the  review  had  on 
outcomes  of  projects.  There  is  a  strong  negative  relationship  between  the  rankings  of  the  variables, 
which  is  statistically  significant  at  99%  confidence. 


Q34  When  was  the  last  peer  review  of  an  S&T  activity  in  your  unit?  *  Q39  In 
general,  how  much  impact  does  peer  review  have  on  the  outcomes  of  your 

projects? 

In  general,  how  much  impact  does 
peer  review  have  on  the  outcomes 
of  your  projects? 

Total 

No 

Impact 

Some 

Impact 

Significant 

Impact 

When  was  One  year  ago 

1 

10 

1 

12 

the  last  peer  years  ago 

0 

2 

0 

2 

review  of  an  Three  vears  aoo 

S&T  activity  ^  ^ 

2 

1 

0 

3 

in  your  unit?  Four  years  ago 

1 

2 

0 

3 

Five  years  ago 

0 

1 

0 

1 

More  than  five 

years  ago 

3 

0 

0 

3 

Total 

7 

16 

1 

24 

(190) 


Q34  When  was  the  last  peer  review  of  an  S&T  activity  in  your  unit?  * 
Q39  In  general,  how  much  impact  does  peer  review  have  on  the 
outcomes  of  your  projects? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal 

Kendall's  tau-c 

-0.438 

0.001 

Nominal  by  Nominal 

Cramer's  V 

0.536 

0.183 

(191) 
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It  appears  that  there  is  a  relationship  between  how  valuable  an  S&T  feedback  is  and  the  impact  of 
feedback  on  a  unit’s  capabilities  and  competencies;  in  other  words,  feedback  that  is  considered 
valuable  tends  to  be  incorporated  into  the  unit’s  activities.  There  is  a  strong  positive  relationship 
between  the  rankings  of  the  variables,  which  is  statistically  significant  at  95%  confidence. 


Q36  Overall,  did  your  last  peer  review  offer  valuable  feedback?  *Q37  In  general,  how  much 
impact  does  peer  review  have  on  your  unit's  capabilities  and  competencies? 

In  general,  how  much  impact  does  peer 
review  have  on  your  unit's  capabilities 
and  competencies? 

Total 

No  Impact 

Some 

Impact 

Significant 

Impact 

Overall,  did  your  last  Not  Valuable 

1 

1 

0 

2 

peer  review  offer 

2 

3 

0 

5 

valuable  feedback? 

Valuable 

1 

13 

3 

17 

Total 

4 

17 

3 

24 

(192) 


Q36  Overall,  did  your  last  peer  review  offer  valuable  feedback?  *Q37 

In  general,  how  much  impact  does  peer  review  have  on  your  unit's 
capabilities  and  competencies? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal  Kendall's  tau-b 

Nominal  by  Nominal  Cramer's  V 

0.438 

0.344 

0.018 

0.225 

(193) 

It  appears  that  there  is  a  relationship  between  how  valuable  an  S&T  feedback  is  and  the  impact  of 
feedback  on  a  unit’s  project  outcomes;  in  other  words,  feedback  that  is  considered  valuable  tends  to 
be  incorporated  into  the  unit’s  project  management.  There  is  a  very  strong  positive  relationship 
between  the  rankings  of  the  variables,  which  is  statistically  significant  at  99.9%  confidence. 


Q36  Overall,  did  your  last  peer  review  offer  valuable  feedback?  *  Q39  In  general,  how 
much  impact  does  peer  review  have  on  the  outcomes  of  your  projects? 

In  general,  how  much  impact  does  peer 
review  have  on  the  outcomes  of  your 
projects? 

No  Impact 

Some  Impact 

Significant 

Impact 

Total 

Overall,  did  your  last 
peer  review  offer 

Not 

Valuable 

1 

1 

0 

2 

valuable  feedback? 

Neutral 

5 

0 

0 

5 

Valuable 

1 

15 

1 

17 

Total 

7 

16 

1 

24 

(194) 
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Q36  Overall,  did  your  last  peer  review  offer  valuable  feedback?  *  Q39  In 
general,  how  much  impact  does  peer  review  have  on  the  outcomes  of  your 

projects? 


Value 

Approx.  Sig. 

Ordinal  by  Ordinal 

Kendall's  tau-b 

0.690 

0.000 

Nominal  by  Nominal 

Cramer's  V 

0.596 

0.002 

(195) 


Where  there  is  quantitative  impact  {how  much...)  on  project  outcomes,  that  impact  tends  to  be 
positive  in  quality  {what  kind...).  There  is  a  very  strong  and  positive  relationship  between  these 
variables,  which  is  statistically  significant  at  99.9%  confidence 


Q39  In  general,  how  much  impact  does  peer  review  have  on  the  outcomes  of  your 

projects?  *  Q40  In  general,  what  kind  of  impact  does  peer  review  have  on  the 

outcomes  of  your  projects? 

In  general,  what  kind  of  impact  does 
peer  review  have  on  the  outcomes  of 

your  projects? 

Neutral/  No 

Impact 

Positive  Impact 

Total 

In  general,  how  much 

No  Impact 

7 

0 

7 

impact  does  peer 
review  have  on  the 

Some 

outcomes  of  your 

Impact 

7 

9 

16 

projects? 

Significant 

Impact 

0 

1 

1 

Total 

14 

10 

24 

(196) 


Q39  In  general,  how  much  impact  does  peer  review  have  on  the 
outcomes  of  your  projects?  *  Q40  In  general,  what  kind  of  impact 
does  peer  review  have  on  the  outcomes  of  your  projects? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal  Kendall's  tau-c 

Nominal  by  Nominal  Cramer's  V 

0.535 

0.570 

0.000 

0.020 

(197) 
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Where  there  is  quantitative  impact  on  the  unit’s  capabilities  and  competencies  that  impact  tends  to 
be  positive  in  quality.  There  is  a  moderate-strong  positive  relationship  between  these  variables, 
which  is  statistically  significant  at  95%  confidence. 


Q38  In  general,  what  kind  of  impact  does  peer  review  have  on  your  unit's  capabiiities  and 
competencies?  *  Q37  in  generai,  how  much  impact  does  peer  review  have  on  your  unit's 

capabiiities  and  competencies? 

In  general,  how  much  impact  does  peer  review 
have  on  your  unit's  capabilities  and 
competencies? 

No  Impact 

Some  Impact 

Significant 

Impact 

Total 

In  general,  what  kind  of  Neutral/ 
impact  does  peer  No 

3 

5 

0 

8 

review  have  on  your  Impact 

unit's  capabilities  and  positive 

competencies? 

1 

12 

3 

16 

Total 

4 

17 

3 

24 

(198) 


Q38  In  general,  what  kind  of  impact  does  peer  review  have  on  your  unit's 
capabilities  and  competencies?  *  Q37  In  general,  how  much  impact  does 
peer  review  have  on  your  unit's  capabilities  and  competencies? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal  Kendall's  tau-c 

Nominal  by  Nominal  Cramer's  V 

0.382 

0.445 

0.015 

0.093 

(199) 
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Respondents  reporting  a  quantitative  impact  of  S&T  reviews  on  project  outcomes  tend  to  also  report 
a  quantitative  impact  on  the  unit’s  competencies  and  capabilities.  There  is  a  very  strong  positive 
relationship  between  the  ranking  of  these  two  variables,  which  is  statistically  significant  at  99% 
confidence. 


Q37  In  general,  how  much  impact  does  peer  review  have  on  your  unit's  capabiiities  and 
competencies?  *  Q39  in  generai,  how  much  impact  does  peer  review  have  on  the  outcomes  of  your 

projects? 

In  general,  how  much  impact  does  peer  review  have 
on  the  outcomes  of  your  projects? 

Total 

No  Impact 

Some  Impact 

Significant 

Impact 

In  general,  how  much 

No  Impact 

A 

n 

n 

A 

impact  does  peer 

review  have  on  your 

Some 

unit's  capabilities  and 

Impact 

3 

14 

0 

17 

competencies? 

Significant 

0 

2 

\ 

3 

Impact 

Total 

7 

16 

1 

24 

(200) 


Q37  In  general,  how  much  impact  does  peer  review  have  on  your 
unit's  capabilities  and  competencies?  *  Q39  In  general,  how  much 
impact  does  peer  review  have  on  the  outcomes  of  your  projects? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal  Kendall's  tau-b 

Nominal  by  Nominal  Cramer's  V 

0.684 

0.626 

0.001 

0.001 

(201) 


Collaborative  Relationships 

Two-thirds  of  respondents  (66.6%)  rate  their  collaborative  working  relationship  with  the  private 
sector  as  either  ‘Good’  or  ‘Excellent’;  a  somewhat  lower  number  (58%)  rate  their  collaborative 
working  relationship  with  the  academic  community  similarly. 


Q41  How  would  you  rate  your  unit's  collaborative  working  relationship 
with  the  private  sector  S&T  community? 

Frequency 

Valid  Percent 

Valid 

Poor 

8 

15.7 

Fair 

9 

17.6 

Good 

22 

43.1 

Excellent 

12 

23.5 

Total 

51 

100.0 

(202) 
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Q42  How  would  you  rate  your  unit's  collaborative  working  relationship 
with  the  academic  community? 

Frequency 

Valid  Percent 

Valid 

Poor 

5 

10.0 

Fair 

16 

32.0 

Good 

22 

44.0 

Excellent 

7 

14.0 

Total 

50 

100.0 

(203) 

There  appears  to  be  a  relationship  between  how  respondents  rate  their  collaborative  relationships 
with  private  sector  and  academic  partners.  Respondents  who  report  having  a  positive  relationship 
with  one  group  tend  to  also  have  a  positive  relationship  with  the  other  group.  There  is  a  moderate- 
strong  positive  relationship  between  the  ranking  of  these  two  variables,  which  is  statistically 
significant  at  95%  confidence. 


Q41  How  would  you  rate  your  unit's  collaborative  working  relationship  with  the  private  sector  S&T 

community?  *  Q42  How  would  you  rate  your  unit's  collaborative  working  relationship  with  the 

academic  community? 

How  would  you  rate  your  unit's  collaborative  working 

relationship  with  the  academic  community? 

Poor 

Fair 

Good 

Excellent 

Total 

How  would  you  rate  your  Poor 
unit's  collaborative  . 

5 

3 

0 

0 

8 

working  relationship  with 

0 

2 

5 

2 

9 

the  private  sector  S&T  Good 

community? 

0 

8 

11 

2 

21 

Excellent 

0 

3 

5 

3 

11 

Total 

5 

16 

21 

7 

49 

(204) 


Q41  How  would  you  rate  your  unit's  collaborative  working  relationship  with 
the  private  sector  S&T  community?  *  Q42  How  would  you  rate  your  unit's 
collaborative  working  relationship  with  the  academic  community? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal  Kendall's  tau-b 

Nominal  by  Nominal  Cramer's  V 

0.355 

0.479 

0.007 

0.000 

(205) 
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Very  few  respondents  (26.3%)  were  aware  of  the  results  of  their  units  most  recent  client  satisfaction 
survey.  However,  over  half  of  respondents  (52%)  claim  they  ‘Often’  or  ‘Always’  receive  feedback 
directly  from  clients  or  partners  after  a  product  enters  service.  The  degree  of  candour  and  the 
breadth  of  feedback  may  be  limited  in  direct  feedback. 


Q43  Were  you  informed  of  the  results  of  the  most  recent 
partner/  client  satisfaction  survey  for  your  unit? 

Frequency 

Valid  Percent 

Valid  Yes 

10 

26.3 

No 

28 

73.7 

Total 

38 

100.0 

(206) 


Q44  How  often  do  you  receive  feedback  directly  from  clients  or  partners 
after  S&T  products  enter  service? 

Frequency 

Valid  Percent 

Valid  Never 

1 

1.9 

Rarely 

7 

13.5 

Sometimes 

17 

32.7 

Often 

24 

46.2 

Always 

3 

5.8 

Total 

52 

100.0 

(207) 

There  is  a  moderate  relationship,  which  is  significant  at  95%  confidence,  between  how  often  a 
respondent  receives  direct  feedback  and  if  they  had  been  informed  of  the  most  recent  client  survey 
results.  In  other  words  the  more  often  a  manager  receives  feedback  directly  from  a  client  the  more 
likely  that  manager  is  to  be  aware  of  the  most  recent  client  satisfaction  survey  results. 


Q44  How  often  do  you  receive  feedback  directly  from  clients  or  partners  after  S&T 

products  enter  service? 

*  Q43  Were  you  informed  of  the  results  of  the  most  recent  partner/ 

client  satisfaction  survey  for  your  unit? 

Were  you  informed  of  the  results  of  the 
most  recent  partner/  client  satisfaction 

survey  for  your  unit? 

Yes 

No 

Total 

How  often  do  you 

Never 

0 

1 

1 

receive  feedback 

Rarely 

0 

6 

6 

directly  from  clients  or 
partners  after  S&T 

Sometimes 

3 

8 

11 

products  enter  service? 

Often 

5 

11 

16 

Always 

1 

0 

1 

Total 

9 

26 

35 

(208) 
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Q44  How  often  do  you  receive  feedback  directly  from  clients  or  partners 
after  S&T  products  enter  service?  *  Q43  Were  you  informed  of  the  results 
of  the  most  recent  partner/  client  satisfaction  survey  for  your  unit? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal  Kendall's  tau-c 

Nominal  by  Nominal  Cramer's  V 

-0.291 

0.399 

0.038 

0.233 

(209) 

There  are  no  relationships  between  how  often  direet  feedback  is  received  from  clients  and  the 
quality  of  the  collaborative  relationship  with  either  the  private  sector  or  academic  communities. 

Sixty  five  percent  (65.4%)  of  respondents  report  that  feedback  from  clients  ‘Often’  or  ‘Always’ 
influences  the  design  or  delivery  of  future  projects. 


Q45  Does  feedback  from  partners  and  clients  influence  the  design 
or  delivery  of  future  projects? 

Frequency 

Valid  Percent 

Valid  Rarely 

2 

3.6 

Sometimes 

17 

30.9 

Often 

24 

43.6 

Always 

12 

21.8 

Total 

55 

100.0 

(210) 

There  is  a  moderate  positive  relationship  between  how  often  managers  receive  direct  feedback  from 
clients  and  how  often  feedback  influences  future  projects.  This  relationship  is  statistically  significant 
at  95%  confidence. 


Q44  How  often  do  you  receive  feedback  directly  from  clients  or  partners  after  S&T  products  enter 
service?  *  Q45  Does  feedback  from  partners  and  clients  influence  the  design  or  delivery  of  future 

projects? 

Does  feedback  from  partn 
design  or  deliver 

ers  and  clients  influence  the 
/  of  future  projects? 

Total 

Rarely 

Sometimes 

Often 

Always 

How  often  do  you 

Never 

0 

1 

0 

0 

1 

receive  feedback 

Rarely 

0 

4 

3 

0 

7 

directly  from  clients  or 

Sometimes 

O 

R 

A 

A 

partners  after  S&T 

products  enter  service? 

Often 

0 

5 

13 

6 

24 

Always 

0 

0 

2 

1 

3 

Total 

2 

16 

22 

11 

51 

(211) 
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Q44  How  often  do  you  receive  feedback  directly  from  clients  or  partners  after  S&T 
products  enter  service?  *  Q45  Does  feedback  from  partners  and  clients  influence 
the  design  or  delivery  of  future  projects? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal  Kendall's  tau-c 

0.274 

0.002 

Nominal  by  Nominal  Cramer's  V 

0.305 

0.289 

(212) 


Collaboration  on  Policy  and  Programs  Development 

Most  policy  and  program  development  done  collaboratively  would  be  with  other  employees  of 
DRDC. 

In  general,  60.9%  respondents  describe  the  frequency  of  their  collaboration  specifically  on  policy 
and  program  development  as  ‘Often’  or  ‘Always’;  and  31.3%  describe  the  frequency  as 
‘Sometimes’  . 


Q46  How  often  do  you  collaborate  with  others  when 
developing  policies  or  programs? 

Frequency 

Valid  Percent 

Valid  Never 

1 

1.6 

Rarely 

4 

6.3 

Sometimes 

20 

31.3 

Often 

29 

45.3 

Always 

10 

15.6 

Total 

64 

100.0 

(213) 

Nearly  twenty  percent  (19.0%)  of  respondents  describe  their  overall  quality  of  their  collaborative 
relationships  specifically  dealing  with  policy  and  program  development  as  ‘Excellent’,  and  another 
68.3%  describe  it  as  ‘Good’. 


Q46a  In  general,  how  would  you  assess  the  quality  of  your 
collaborative  relationships? 

Frequency 

Valid  Percent 

Valid  Fair 

8 

12.7 

Good 

43 

68.3 

Excellent 

12 

19.0 

Total 

63 

100.0 

(214) 
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Three  quarters  of  respondents  (76.2%)  report  that  their  collaborative  relationships  specifically 
dealing  with  policy  and  program  development  achieved  most  of  the  intended  goals. 


Q46b  To  what  extent  did  your  most  recent  collaboration 
achieve  its  intended  goals  or  requirements? 

Frequency 

Valid  Percent 

Valid  Achieved  some 

11 

17.5 

Achieved  most 

48 

76.2 

Achieved  all 

4 

6.3 

Total 

63 

100.0 

(215) 

It  appears  that  the  more  positive  a  collaborative  relationship  is,  the  more  likely  the  effort  is  to  have 
achieved  its  goals.  There  is  a  strong  positive  relationship  between  the  rankings  of  the  two  variables, 
which  is  statistically  significant  at  99%  confidence. 


Q46a  In  general,  how  would  you  assess  the  quality  of  your  collaborative  relationships?  *  Q46b  To 
what  extent  did  your  most  recent  collaboration  achieve  its  intended  goals  or  requirements? 


To  what  extent  did  your  most  recent  collaboration 
achieve  its  intended  goals  or  requirements? 

Total 

Achieved  some 

Achieved  most 

Achieved  all 

Fair 

7 

1 

0 

8 

Good 

3 

37 

3 

43 

Excellent 

1 

10 

1 

12 

11 

48 

4 

63 

In  general,  how  would 
you  assess  the  quality 
of  your  collaborative 
relationships? 

Total 


(216) 


Q46a  In  general,  how  would  you  assess  the  quality  of  your 
collaborative  relationships?  *  Q46b  To  what  extent  did  your  most 
recent  collaboration  achieve  its  intended  goals  or  requirements? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal  Kendall's  tau-b 

Nominal  by  Nominal  Cramer's  V 

0.418 

0.498 

0.006 

0.000 

(217) 
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DRDC  Mentoring  Program 

Sixty  percent  (60%)  of  respondents  have  been  involved  in  the  DRDC  Mentoring  Program  at  some 
point,  most  as  Mentors  (69%). 


Q59  Have  you  been  involved  in  the  DRDC  Mentoring  Program? 

Frequency 

Valid  Percent 

Valid  Yes 

42 

60.0 

No 

28 

40.0 

Total 

70 

100.0 

(218) 


Q59a  Have  you  been  involved  in  the  DRDC 
Mentoring  Program? 

As  a  Mentor 

69% 

As  a  Mentee 

40.5% 

As  a  committee  member 

9.5% 

(219) 

Most  respondents  rated  their  experience  with  the  Mentoring  Program  as  either  ‘Good’  (40.5%)  or 
‘Excellent’  (40.5%) 


Q59b  How  would  you  rate  your  satisfaction  with  the  DRDC  Mentoring 

Program? 

Frequency 

Valid  Percent 

Valid  Poor 

2 

4.8 

Fair 

6 

14.3 

Good 

17 

40.5 

Excellent 

17 

40.5 

Total 

42 

100.0 

(220) 

Managers  who  had  not  been  involved  in  the  Mentoring  Program  were  asked  about  their  impression 
of  the  program;  many  held  ‘No  Opinion’  of  the  Program  (57.1%),  while  some  non-participants  have 
a  positive  impression  of  the  program  (39.3%). 


Q59c  What  is  your  impression  of  the  DRDC  Mentoring  Program? 

Frequency 

Valid  Percent 

Valid  Positive 

11 

39.3 

No  Opinion 

16 

57.1 

Negative 

0 

0.0 

Unaware  of  Program 

1 

3.6 

Total 

28 

100.0 

(221) 
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All  respondents  were  asked  if  they  would  be  interested  in  participating  in  the  Mentoring  Program  in 
the  future.  Many  respondents  are  unsure  (27.1%). 


Q60  Are  you  interested  in  participating  in  the  DRDC  Mentoring  Program  in 

the  future? 

Frequency 

Valid  Percent 

Valid  Yes 

39 

55.7 

No 

12 

17.1 

Unsure 

19 

27.1 

Total 

70 

100.0 

(222) 

By  analyzing  future  interest  in  program  participation  against  past  participation  in  the  program,  it  can 
be  concluded  that  those  who  participated  in  the  Mentoring  program  in  the  past  appear  more  likely  to 
participate  in  the  program  in  the  future.  Statistical  testing  shows  a  strong  relationship  between  the 
two  variables,  which  is  significant  at  99.9%  confidence. 


Q59  Have  you  been  involved  in  the  DRDC  Mentoring  Program?  *  Q60  Are  you  interested  in 
participating  in  the  DRDC  Mentoring  Program  in  the  future? 

Q60  Are  you  interested  in 
participating  in  the  DRDC 
Mentoring  Program  in  the  future? 

Yes 

No/  Unsure 

Total 

Q59  Have  you  been 

Yes 

33 

9 

42 

involved  in  the  DRDC 
Mentoring  Program? 

No 

6 

22 

28 

Total 

39 

31 

70 

(223) 


Q59  Have  you  been  involved  in  the  DRDC  Mentoring  Program?  *  Q60  Are  you 
interested  in  participating  in  the  DRDC  Mentoring  Program  in  the  future? 

Value 

df 

Asymp.  Sig. 
(2-sided) 

Approx.  Sig. 

Pearson  Chi-Square 

Phi 

22.233 

0.564 

1 

0.000 

0.000 

(224) 
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For  current  and  past  participants  there  is  a  moderate-strong  relationship  between  satisfaction  with 
their  experience  and  their  interest  in  future  participation. 


Q60  Are  you  interested  in  participating  in  the  DRDC  Mentoring  Program  in  the  future?  *  Q59b  How 
wouid  you  rate  your  satisfaction  with  the  DRDC  Mentoring  Program? 

How  would  you  rate  your  satisfaction  with  the  DRDC 
Mentoring  Program? 

Poor 

Fair 

Good 

Excellent 

Total 

Are  you  interested  in  Yes 

participating  in  the 

DRDC  Mentoring 

1 

3 

13 

16 

33 

0 

1 

0 

0 

1 

Program  in  the  future?  Unsure 

1 

2 

4 

1 

8 

Total 

2 

6 

17 

17 

42 

(225) 


Q60  Are  you  interested  in  participating  in  the  DRDC  Mentoring  Program  in 
the  future?  *  Q59b  How  wouid  you  rate  your  satisfaction  with  the  DRDC 

Mentoring  Program? 

Value 

Approx.  Sig. 

Ordinal  by  Ordinal 

Kendall's  tau-c 

-0.243 

0.013 

Nominal  by  Nominal 

Cramer's  V 

0.358 

0.097 

(226) 

The  majority  of  respondents  interested  in  participating  in  the  Mentoring  Program  in  the  future  would 
elect  to  participate  as  Mentors  (94.9%),  although  a  significant  number  are  also  interested  in 
participating  as  Mentees  (30.8%)  or  as  a  member  of  the  Program  committee  (28.2%). 


Q60a  Are  you  interested  in  participating  in  the 
DRDC  Mentoring  Program  in  the  future? 

As  a  Mentor 

94.9% 

As  a  Mentee 

30.8% 

As  a  committee  member 

28.2% 

(227) 

There  were  22  comments  made  by  respondents  concerning  the  DRDC  Mentoring  Program.  Below  is 
a  table  summarizing  the  comments. 


Comment 

Frequency 

General  positive  experience  comment 

10 

Program  is  not  effectively  advertised 

4 

Not  enough  time  to  be  in  the  program 

4 

Need  for  better  vetting  process  in  selection  of  participants 

2 

Unsure  who  is  eligible  for  the  program 

2 

Participates  in  DND  Mentoring  Program 

2 

Other 

1 

(228) 
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General  Comments 


A  number  of  respondents  commented  on  the  design  and  implementation  of  the  survey.  Some  of 
these  comments  seem  to  reflect  a  level  of  survey  fatigue  within  the  organization. 

“This  survey  was  rather  long.  A  progress  bar  would  be  appreciated.” 

“1  am  seeing  a  number  of  requests  for  surveys.  It  is  not  always  clear  to  me  that  the  reminder 
notice  is  for  a  new  survey  or  one  which  1  have  already  completed.” 

Some  respondents  had  difficulty  answering  questions,  often  due  to  their  own  particular  situation,  or 
that  of  their  Centre. 

“1  had  difficulty  applying  some  of  the  questions  to  my  particular  case,  so  1  adopted  a  best-fit 
strategy.” 

“Many  of  the  questions  were  not  taking  into  account  the  different  ways  that  scientific  work  is 
done  in  DRDC” 

There  were  both  positive  and  negative  comments  about  collaborative  relationships  with  clients  and 
partners.  One  respondent  felt  there  was  a  long  history  of  positive  collaborative  relationships  in  the 
organization: 

“We. ..have  extensive  experience  in  working  with  CF/DND  clients  and  stakeholders  [and]  we  have 
been  doing  this  well  for  years.” 

However,  another  respondent  pointed  out  the  danger  of  collaborative  relationships  when  the 
individual  becomes  aligned  with  the  stakeholder  or  client  at  the  expense  of  the  organization: 

“Certains  des  employes  sont  plus  loyaux  au  client  qu'a  RDDC.” 

Some  comments  addressed  the  need  to  formally  document  policies  and  duties  at  DRDC: 

“[Nous  sommes]  en  train  de  mettre  en  place  des  instruments  expliquant  le  role  et  les 
responsabilites  de  chacun  et  pour  mieux  planifier  le  programme  et  I'allocation  des  ressources.” 
(We  are  developing  explanations  of  everyone’s  roles  in  improving  program  planning  and 
resource  allocation.) 

“We  need  an  accommodation  plan  ...  that  is  open  and  transparent  to  all  employees.” 

“There  should  be  an  open  annual  report  on  the  progress  and  success  of  the  DRDC  Mentoring 
Program.” 
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8  Findings 


The  following  findings  are  based  on  the  analyses  of  the  data  from  surveys  2  and  3.  These  findings 
present  base-line  organizational  information  for  the  six  core  elements  of  the  MAP,  and  demonstrate 
the  utility  of  the  MAP  for  organizational  development,  as  well  as  the  feasibility  of  the  survey 
instrument  for  gathering  information  not  readily  available  elsewhere  in  the  Agency. 

bindings  are  organized  by  element  and  are  followed  in  each  case  by  an  assessment  of  the  elemental 
Indicators,  based  only  on  survey  data  and  corporate  information  and  documentation  already 
referenced  in  this  report,  and  modeled  on  the  TB  assessment  criteria  shown  in  Table  24.  This  raises 
an  important  distinction  between  the  TB  and  DRDC  MAPs.  The  TB  MAP  generally  examines 
policies  and  frameworks,  whereas  the  DRDC  MAP,  through  its  survey,  examines  actual 
performance.  It  is  important  to  remember  that  the  survey  instrument  is  only  one  of  possible  multiple 
measurement  instruments  for  each  element,  thus  the  results  of  the  survey  must  be  weighed  against 
other  element  measures  appropriately.  Our  analysis  will  assess  DRDC  MAP  performance  almost 
exclusively  from  the  perspective  of  the  survey. 

While  the  TB  MAP  assesses  departmental  performance  based  on  the  existence  of  policies  and 
frameworks,  by  utilizing  a  survey  instrument,  the  DRDC  MAP  is  adding  a  new  dimension  to  the 
MAP.  Managerial  self-reporting  reveals  how  policies  and  frameworks  are  actually  utilized  within 
the  organization.  As  such,  when  compared  to  TB  MAP  assessment,  DRDC  MAP  assessments  will 
generally  be  more  critical. 


The  DRDC  MAF  evaluates  the  walk,  not  the  talk 
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Table  24:  MAP  Assessment  Scale  -  Round 


Attention 

Required 

•  Significant  deficiencies,  or  deficiencies  in  most  of  the  measures  listed  for 
the  indicator 

and/or 

•  Inadequate  attention  to  the  deficiencies 

Opportunity  for 
Improvement 

•  Moderate  deficiencies,  or  deficiencies  in  some  of  the  measures  listed  for 
the  indicator 

and 

•  Evidence  of  attention  to  the  deficiencies  and  progress 

Acceptable 

•  No  significant  deficiencies  in  any  of  the  measures 
and 

•  Meets  Treasury  Board  Policy  expectations 

Strong 

•  No  deficiencies  in  any  of  the  measures 
and 

•  Sustained  performance  for  the  indicator  that  exceeds  Treasury  Board 

Policy  expectations  and  suggests  continued  strong  performance 

Insufficient 

Information 

•  Neither  Treasury  Board  Policy  nor  the  department  currently  has  the 
information  required  for  the  indicator  (for  example,  because  the  indicator 
is  new  or  due  to  very  recent  organizational  changes). 

Not  Applicable 

•  Does  not  apply  to  the  Department  /  Agency 

8.1  Policy  and  Programs 

Indicator:  Sustained  analytic  capacity  and  culture  of  consultation,  review  and 
challenge 

Development  of  policy  capacity,  which  for  DRDC  includes  S&T  capacity,  is  often  problematic  in 
DRDC.  Many  managers  feel  that  key  positions  are  not  filled  quickly  enough,  though  once  they  are, 
the  individual  performs  effectively  in  a  reasonable  timeframe.  Additionally,  there  is  no  relationship 
between  the  identification  of  high  performing  employees  and  how  quickly  a  key  position  is  filled, 
nor  how  effectively  a  new  employee  in  a  key  position  performs.  This  may  be  a  reflection  of  lack  of 
succession  planning.  Generally,  despite  the  lack  of  a  succession  plan  in  the  organization,  managers 
are  identifying  high  potential  employees  in  their  units  and  seem  to  feel  that  there  are  employees 
ready  to  compete  for  key  positions.  As  may  be  expected  in  an  S&T  based  environment,  most 
managers  report  that  keeping  up  with  new  ideas,  methods  and  technology  is  very  important  to  their 
job  performance;  however,  there  are  mixed  reports  as  to  the  satisfaction  with  resources  available  to 
do  so. 


Treasury  Board  of  Canada  Secretariat,  MAF  Assessment  Scale  -  Round  IV,  available  from:  httri://www.tbs- 
sct.gc.ca/maf-crg/assessments-evaluations/2006/scale-echelle-eng.asp. 
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In  terms  of  planning  for  future  capacity  in  personnel,  there  are  some  encouraging  findings.  The 
DRDC  Mentoring  Program  offers  an  opportunity  to  support  knowledge  transfer.  Most  participants 
in  the  mentoring  program  have  had  positive  experiences,  and  would  participate  in  the  program  in  the 
future.  Non-participants  appear  to  have  limited  knowledge  about  the  program,  and  little  desire  to 
participate.  Subordinates’  Personal  Learning  Plans  (PLP’s)  are  linked  to  Knowledge,  Skill  and 
Abilities  (KSA’s)  for  current  and  future  roles;  and  the  number  of  managers  encouraging  cross 
training  and  knowledge  transfer  through  team  work  is  encouraging.  The  employment  of  students 
appears  to  be  successful  in  that  those  who  have  worked  with  student  employees  in  the  past  have  had 
positive  experiences,  although  we  have  no  data  on  the  number  of  students  employed  nor  the  quality 
of  experiences  from  the  students’  perspectives. 

Peer  Review  is  an  important  factor  in  maintaining  policy  quality.  Most  S&T  peer  reviews  appear  to 
fall  into  the  five  year  cycle,  although  some  respondents  say  their  unit  has  never  had  a  review  or  that 
the  last  review  was  over  5  years  ago.  Most  report  having  had  positive  outcomes  and  valuable 
feedback  from  their  review.  S&T  review  is  more  likely  to  affect  a  unit’s  capabilities  and 
competencies  than  it  affects  project  outcomes;  however,  there  seems  to  be  little  quantity  or  quality 
of  impact  on  a  unit’s  projects  or  capabilities  and  competencies  due  to  an  S&T  review,  as 
implementation  of  review  outcomes  appears  to  be  optional  rather  than  mandatory. 

Few  managers  are  aware  of  the  results  of  client  and  partner  satisfaction  surveys,  and  few  receive 
direct  feedback  from  clients  after  a  product  enters  service.  This  results  in  an  incomplete  policy- 
feedback  loop,  using  Eastman’s  policy  model. 

The  infrastructure  capacity  of  this  element  is  addressed  through  the  Stewardship  Element. 

Indicator:  Results  focused  policy  and  program  agendas  aligned  with 
government’s  horizontal  priorities 

According  to  survey  responses,  the  strategic  goals  of  Canada’s  national  S&T  strategy^®'*  appear  to  be 
adequately  represented  in  DRDC’s  programs,  with  most  activities  addressing  multiple  national 
goals.  The  workforce  development  goal  of  the  national  strategy  is  one  that  is  not  as  well  represented, 
largely  because  DRDC’s  commercialization  and  mission  directed  R&D  activities  require  a  level  of 
expertise  which  is  not  conducive  to  many  conventional  workforce  development  goals.  As  such, 
DRDC  will  need  to  make  special  efforts  to  support  workforce  development  outside  of 
commercialization  and  mission  directed  R&D  activities. 

Indicator:  Social  Evaluation  of  Impacts 

A  mechanism  for  identifying  stakeholders  outside  of  the  partner/client  paradigm  does  not  currently 
exist  at  DRDC. 

Indicator:  Confidence  of  the  ADM  (S&T),  clients  and  partners 

This  indicator  was  not  assessed  in  our  survey. 

Assessment:  Policy  and  Programs 


Industry  Canada,  Mobilizing  Science  and  Technology  to  Canada ’s  Advantage. 
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Table  25,  below,  shows  the  assessment  rating  of  each  Indicator  and  the  overall  assessment  for  the 
element,  based  on  survey  findings. 


Table  25:  DRDC  MAF  Assessment,  Policy  and  Programs 


Indicator 

Assessment 

Sustained  analytic  capacity  and  culture  of 
consultation,  review  and  challenge 

Opportunity  for  Improvement 

Results  focused  policy  and  program  agendas 
aligned  with  government’s  horizontal  priorities 

Strong 

Social  Evaluation  of  Impacts 

Attention  Required 

Confidence  of  the  ADM  (S&T),  clients  and 
partners 

Insufficient  Information 

OVERALL 

Opportunity  for  Improvement 

8.2  People 

Although  the  Public  Service  Employee  Survey^*"  is  the  preferred  survey  instrument  for  the  People 
element  of  the  DRDC  MAF,  several  relevant  questions  were  asked  of  Level  2  managers  during  the 
third  pilot  survey  under  the  guise  of  the  Policy  and  Programs  element.  Also,  a  number  of  relevant 
People  Element  conditions  are  addressed  through  the  Policy  and  Programs  element,  as  workforce 
planning  and  capacity  are  integral  components  of  that  element. 

Indicator:  Workforce  is  sufficient  in  numbers  and  abilities  while  reflecting  the 
Canadian  population 

Neither  the  population  nor  the  response  rate  for  Level  2  managers  support  confidence  in  the  results; 
nonetheless,  only  one -third  of  the  six  Level  2  respondents  felt  that  employment  equity  groups  were 
adequately  represented  in  feeder  groups  for  key  positions.  This  is  consistent  with  statistics  on 
employment  equity  group  representation  reported  in  DRDC’s  Annual  Report.  Half  of  the  Level  3 
respondents  reported  offering  developmental  opportunities  specifically  for  employment  equity 
groups. 

This  indicator  was  also  addressed  through  policy  capacity  issues  in  the  Policy  and  Programs 
Element  (see  above).  Also,  our  survey  showed  that  80%  of  Level  3  managers  report  that  their  unit 
does  not  have  sufficient  FTE  to  fulfill  its  mandate.  This  is  not  necessarily  cause  for  alarm  in  and  of 
itself;  an  audit  of  workload  slippage  and  other  unit  performance  criteria  would  provide  a  better 
indication  of  adequate  staffing  levels. 

The  lack  of  an  Agency- wide  Succession  Plan  has  been  noted  in  this  report.  Defence  R&D  Canada’s 
mentoring  program  offers  one  component  of  a  successful  succession  plan. 


201  2008  Public  Service  Employee  Survey  Results. 

DRDC,  Annual  Report  for  the  year  ending  3 1  March  2008,  p.  6 1 . 
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Indicator:  Workplace  is  a  supportive  and  respectful  environment,  including 
respect  for  official  language  requirements 

This  indicator  was  not  assessed  in  our  survey. 

Indicator:  All  levels  of  management  demonstrate  effective  leadership 

This  indicator  was  not  assessed  directly  in  our  survey. 

Indicator:  Employees  are  engaged  in  their  work  and  have  opportunities  to 
learn  and  grow 

This  indicator  was  addressed  through  policy  capacity  issues  in  the  Policy  and  Programs  Element.  In 
that  element  we  discovered  that  DRDC  needs  to  make  special  efforts  to  support  workforce 
development  outside  of  commercialization  and  mission  directed  R&D  activities.  Some  efforts  by  the 
Agency  to  address  this  inherent  deficiency  include  the  Mentoring  program,  Personal  Learning  Plans, 
cross  training,  secondment  opportunities  and  team-based  work. 

Assessment:  People 

Table  26,  below,  shows  the  assessment  rating  of  each  Indicator  and  the  overall  assessment  for  the 
element,  based  on  survey  findings. 


Table  26:  DRDC  MAF  Assessment,  People 


Indicator 

Assessment 

Workforce  is  sufficient  in  numbers  and  abilities 
while  reflecting  the  Canadian  population 

Insufficient  Information 

Workplace  is  a  supportive  and  respectful 
environment,  including  respect  for  official 
language  requirements 

Insufficient  Information 

All  levels  of  management  demonstrate 
effective  leadership 

Insufficient  Information 

Employees  are  engaged  in  their  work  and  have 
opportunities  to  learn  and  grow 

Acceptable 

OVERALL 

Insufficient  Information 

186 


DRDC  Atlantic  CR  2009-135 


8.3  Service 

Indicator:  Monitored,  continuousiy  improved  service  quality 

It  is  concerning  that  so  many  managers  appear  to  be  unaware  of  the  results  of  the  most  recent  client 
satisfaction  surveys,  although  this  may  be  in  part  due  to  suspension  of  such  activities  during  stand- 
up  of  Defence  S&T  Enterprise  processes. Overview  Group  or  Thrust  Advisory  Group  records  can 
supplant  missing  client  surveys,  but  cannot  replace  the  candor  of  responses  available  from 
anonymous  client  surveys.  In  absence  of  such  information,  managers’  own  perception  of  service 
quality  may  be  misaligned  with  client  or  partners’  perceptions,  to  the  detriment  of  client/partner 
satisfaction.  Our  survey  finds  that  when  managers  are  aware  of  client  feedback,  it  does  influence  the 
design  and  delivery  of  future  projects. 

On  a  positive  note,  the  EXPEDITION  series  of  organizational  change  initiatives  are  themselves 
indicative  of  an  organization  whose  leadership  is  committed  to  transformation  and  service 
improvement. 

Indicator:  Leading  advice  on  technoiogy  expioitation 

This  indicator  was  not  assessed  in  our  survey. 

Indicator:  Proactive  engagement  of  clients  and  partners 

This  indicator  was  touched  on  by  asking  about  how  project  results  are  communicated.  The  most 
common  communication  methods  were  presentations,  meetings  and  standard  reports.  This  is  in  line 
with  the  publications  and  presentations  data  found  in  the  most  recent  DRDC  Annual  Report. 

Indicator:  Effective  reiationships  with  ciients  &  partners 

There  is  room  for  improvement  in  the  quality  of  collaborative  relationships  with  both  the  private 
sector  and  academic  communities.  The  correlation  between  the  quality  of  these  two  relationships 
suggests  that  individual  attitude  is  a  factor  in  the  success  of  these  collaborative  relationships.  There 
seems  to  be  better  quality  collaborative  relationships  within  the  Agency  than  with  external 
collaborators.  This  seems  reasonable  considering  the  individuals  are  all  coming  from  the  DRDC 
environment.  Our  survey  results  also  indicate  that  higher  quality  collaborative  relationships  lead  to 
increases  in  goal  achievement. 

Assessment:  Service 

Table  27,  below,  shows  the  assessment  rating  of  each  Indicator  and  the  overall  assessment  for  the 
element,  based  on  survey  findings. 


Karim  Dahel,  private  communication. 

DRDC,  “Appendix  4:  Publications  and  Conference  Presentations”  in  Annual  Report  for  the  year  ending  31  March  2008, 


p.  60. 
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Table  27:  DRDC  MAF  Assessment,  Service 


Indicator 

Assessment 

Monitored,  continuously  improved  service 
quality 

Opportunity  for  Improvement 

Leading  advice  on  technology  exploitation 

Insufficient  Information 

Proactive  engagement  of  clients  and  partners 

Insufficient  Information 

Effective  relationships  with  clients  &  partners 

Opportunity  for  Improvement 

OVERALL 

Opportunity  for  Improvement 

8.4  Risk  Management 

Indicator:  Risk  is  managed  within  the  context  of  the  DND  Corporate  Risk 
Profiie 

Risk  management  has  been  problematic  for  DND  in  the  TB  MAF,  and  must  necessarily  be  so  for 
DRDC  in  its  own  MAF.  A  departmental  Corporate  Risk  Profde  is  the  cornerstone  of  the  TBS  Risk 
Management  element.  The  Department  of  National  Defence  has  not  had  a  Corporate  Risk  Profde, 
although  one  is  reportedly  close  to  approval  and  release. 

Indicator:  Risk  Management  is  integrated  with  pianning  and  operations 
structures  throughout  aii  ieveis  of  the  Agency 

There  appears  to  be  no  comprehensive  DRDC  risk  management  profde,  which  should  be  no  surprise 
in  absence  of  a  DND  Corporate  Risk  Profde.  There  is  neither  a  process  in  place  to  identify  risk 
management  stakeholders,  nor  a  communication  plan  to  communicate  with  stakeholders. 

Stakeholder  identification  appears  to  be  based  on  corporate  memory  of  past  consultations  rather  than 
any  data  system  or  structured  process. 

Weakness  in  documenting  project  risks,  coupled  with  weakness  in  risk  reassessment  during  projects 
is  a  major  concern.  It  is  commonly  held  that  risk  management  is  the  purview  of  project  managers, 
rather  than  a  pervasive  responsibility  in  the  organization.  This  project  manager-centric  view  is 
consistent  with  concentration  of  risk  management  activities  in  the  TDP  program.  Outside  of  the  TDP 
program,  the  main  forum  for  consultation  about  risk  issues  related  to  projects  is  TAG  meetings. 

Level  3  Managers  do  not  appear  to  consistently  have  confidence  that  their  subordinates  understand 
their  roles,  responsibilities,  or  limitations  on  authority  when  managing  risks.  In  each  case,  over  half 
of  survey  respondents  either  have  inconsistent  or  no  confidence  that  their  subordinates  understand 
these  risk  management  duties.  Statistical  testing  supports  the  assumption  that  managers  who  make 
their  risk  management  priorities  and  tolerances  clear  to  subordinates  are  more  likely  to  have 
confidence  in  their  subordinates’  understanding  of  their  roles  in  managing  risks. 


An  examination  of  Descartes  revealed  neither  a  policy  statement  nor  an  SOP. 
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Managers  who  report  that  project  plans  consistently  identify  the  likelihood  and  impact  of  risks 
appear  more  likely  to  revisit  risk  assessments  during  the  delivery  of  the  project.  Mangers  who 
report  always  or  often  comparing  progress  to  established  project  milestones  also  appear  somewhat 
more  likely  to  report  always  or  often  identifying  the  likelihood  and  impact  of  project  risks. 

Indicator:  Capacity  exists  to  support  a  Risk  Management  Cuiture 

The  Agency  needs  to  change  the  perception  that  Risk  Management  is  solely  the  responsibility  of 
project  managers,  so  that  each  individual  sees  his/her  own  accountabilities  in  RM.  Managers  who 
consistently  consider  RM  as  a  factor  when  assessing  the  performance  of  subordinates  are  more 
likely  to  have  confidence  in  their  subordinates’  understanding  of  their  roles,  responsibilities  and 
authorities  for  managing  risks  and  have  more  frequent  interaction  with  subordinates  about  RM 
issues.  Integrating  RM  into  Performance  Management  expectations  should  lead  to  increased 
delegation  of  Risk  Management  responsibilities  and  authorities  to  subordinates. 

Assessment:  Risk  Management 

Table  28,  below,  shows  the  assessment  rating  of  each  Indicator  and  the  overall  assessment  for  the 
element,  based  on  survey  findings. 


Table  28:  DRDC  MAF  Assessment,  Risk  Management 


Indicator 

Assessment 

Risk  is  managed  within  the  context  of  the  DND 
Corporate  Risk  Profile 

Attention  Required 

Risk  Management  is  integrated  with  planning 
and  operations  structures  throughout  all  levels 
of  the  Agency 

Opportunity  for  Improvement 

Capacity  exists  to  support  a  Risk  Management 
Culture 

Attention  Required 

OVERALL 

Attention  Required 

8.5  Stewardship 

The  third  pilot  survey  (S3)  revisited  areas  of  interest  from  the  first  pilot  survey  (SI).  Areas  which 
garnered  incomplete  or  unsatisfactory  responses  in  the  first  pilot  were  of  particular  interest.  Overall, 
there  was  little  change  in  responses  from  SI  to  S3,  which  is  not  unexpected  since  there  was  no 
organizational  response  to  the  first  pilot  survey,  as  there  would  be  for  an  operational  MAF. 

Indicator:  Management  systems  provide  reievant  information  and  effective 
feedback  on  resources,  resuits  and  controis 

The  first  pilot  survey  indicated  that  over  half  of  respondents  with  budgeting  responsibilities  had 
budget  tolerances  set  by  their  superior,  and  that  of  those,  all  but  one  individual  successfully 
managed  their  budget  within  that  tolerance.  The  third  pilot  survey  investigated  the  significance  of 
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setting  budget  tolerances,  and  found  that  there  appears  to  be  benefits  to  the  organization.  By  setting 
budget  tolerances  and  supporting  employees  efforts  to  work  within  those  tolerances,  managers  are 
encouraging  delegation  and  a  culture  of  accountability. 

As  in  the  first  pilot  survey,  respondents  in  S3  indicate  that  too  few  units  have  long  term  capital 
plans,  which  are  necessary  for  building  and  maintaining  capacity.  Those  which  do  have  capital 
investment  plans  based  on  long  term  needs  appear  to  be  working  towards  future  capacity  needs 
incrementally.  In  S 1 ,  response  trends  suggested  that  many  managers  view  capital  planning  as  a 
frustrating  exercise,  particularly  when  Vote  5  funding  is  generally  in  short  supply.  There  is  room  for 
improvement  in  the  areas  of  equipment  repair  and  special  maintenance,  and  in  terms  of  ease  of  use 
of  the  request  process,  and  the  timeliness  and  effectiveness  of  the  service. 

The  first  pilot  survey  found  that  there  was  substantial  dissatisfaction  with  the  procurement 
processes,  while  S3  also  determined  that  there  appear  to  be  too  few  procurement  specialists  for  the 
level  of  procurement  activity  in  many  centres.  The  first  pilot  survey  found  that  the  majority  of 
respondents  generally  or  always  had  sound  working  relationships  with  procurement  functional 
authorities;  survey  3  investigated  further  and  found  that,  overall,  the  procurement  process  and  the 
relationships  with  procurement  specialists  appear  to  be  better  at  traditional  research  centres^®®  than 
at  NDHQ-based  centres.  Traditional  research  centres  were  also  clearly  better  than  NDHQ-based 
centres  at  making  progress  of  procurement  files  visible  to  clients,  and  responses  indicate  that  of 
those,  DRDC’s  Atlantic  and  Valcartier  may  be  a  source  of  best  practices. 

Additionally,  in  Survey  2,  a  respondent  noted  that  ineffective  procurement  practices  lead  to 
significant  risk  management  issues  in  terms  of  project  and  program  delivery. 

Indicator:  Effective,  efficient  and  economical  project  management 

There  continues  to  be  a  tendency  to  attribute  project  failures  primarily  to  schedule  and  resource 
issues.  In  the  third  pilot  survey,  as  in  the  first,  it  was  noted  that  documentation  for  projects  managed 
within  a  manager’s  unit  did  not  include  a  responsibility  matrix  or  some  other  formal  assignment  of 
responsibilities. 

Project  management  was  also  reviewed  in  Survey2  as  a  component  of  the  Risk  Management 
element. 

Indicator:  Established  internal  audit  function 

Both  pilot  surveys  found  an  overall  good  level  of  accountability  for  inventory  items.  Both  identified 
areas  of  weakness  in  record  keeping  in  general  and  in  securing  HR  documents  in  particular.  In  both 
surveys,  records  were  reported  as  not  being  sufficiently  complete  for  the  purposes  of  an  audit,  and  in 


^“Atlantic,  Valcartier,  Ottawa,  Toronto,  and  Suffield. 

D.C.  Oxford,  Private  communication:  “DRDC  Atlantic  and  DRDC  Valcartier  are  the  only  centres  using  ORACLE- 
based  applications  to  support  local  administrative  functions,  such  as  procurement.” 
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both  surveys,  personnel  files  are  accessible  to  non-HR  staff  Survey  3  followed  up  on  the 
security  and  tracking  of  classified  documentation,  and  found  some  improvement  over  survey  1 , 
which  may  be  due  to  clarification  of  the  question;  however,  there  is  still  room  for  improvement  in 
the  tracking  of  classified  correspondence. 

Promoting  results  of  activities  through  presentations,  meetings,  and  publications  appears  to  support 
the  DRDC  goals  for  reporting  as  described  in  the  2008  Annual  Report,  appendix  4^°^. 

Indicator:  Compliance  with  policies,  regulations,  and  legislation 

This  indicator  was  not  assessed  in  our  survey. 

Assessment:  Stewardship 

Table  29,  below,  shows  the  assessment  rating  of  each  Indicator  and  the  overall  assessment  for  the 
element,  based  on  survey  findings. 


Table  29:  DRDC  MAF  Assessment,  Stewardship 


Indicator 

Assessment 

Management  systems  provide  relevant 
information  and  effective  feedback  on 
resources,  results  and  controls 

Opportunity  for  Improvement 

Effective,  efficient  and  economical  project 
management 

Opportunity  for  Improvement 

Established  internal  audit  function 

Acceptable 

Compliance  with  policies,  regulations,  and 
legislation 

Insufficient  Information 

OVERALL 

Opportunity  for  Improvement 

8.6  Accountability 

Indicator:  Clear  accountabilities  and  responsibilities  for  due  process  and 
results 

There  is  a  clear  recognition  of  the  principles  of  accountability  by  DRDC  managers,  but  execution  of 
accountability  relationships  is  often  inadequate.  Objective  setting  is  generally  based  upon  the 


^''^Note  of  caution:  the  surveys  asked  managers  “if  access  to  HR  files  was  restricted  to  HR  personnel  only”.  Did  managers 
respond  based  on  personal  experience  of  access  refused  or  granted,  did  they  have  incorrect  understandings  of  policy,  or  did 
their  interpretation  of  ‘access  to  files’  differ  from  that  of  the  survey  designers?  An  example  of  misconception  became 
apparent  during  focus  group-based  testing  of  S3,  when  a  new  manager  was  surprised  to  learn  that  personnel  files  would  not 
be  available  to  him  directly;  instead,  he  would  have  to  obtain  information  from  personnel  files  through  HR  staff  An 
example  of  differing  interpretations  would  be  that  obtaining  information  via  HR  personnel  was  tantamount  to  ‘access’, 
which  was  not  the  survey  designers’  understanding. 

DRDC,  “Appendix  4:  Publications  and  Conference  Presentations”  in  Annual  Report  for  the  year  ending  31  March  2008, 

p.  60. 
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requirements  of  superiors  and  eorporate  objectives,  but  those  objectives  frequently  lack  measurable 
performance  standards,  so  that  it  is  difficult  to  hold  subordinates  to  account.  Level  3  managers  who 
do  set  measurable  performance  standards  for  their  subordinates  also  report  receiving  measurable 
performance  standards  from  their  superiors.  This  reinforces  the  important  role  of  Level  2  managers 
in  establishing  effective  accountability.  Level  2  managers  are  role  models  for  their  subordinates. 

There  is  a  problem  with  Project  documentation:  few  projects  formally  assign  responsibilities.  DRDC 
should  utilize  a  project  responsibility  matrix  for  all  projects,  as  a  responsibility  of  the  PM. 

Indicator:  Delegations  consistent  with  capabilities 

Most  managers  appear  to  follow  Performance  Management  guidelines  when  dealing  with  poor 
performance  by  subordinates,  ranging  from  consultation,  through  developmental  training,  to 
engagement  of  HR  staff;  however  there  are  areas  of  concern,  most  notably  the  practice  of  changing 
goals  to  meet  performance. 

Level  3  managers  appear  to  be  better  at  distinguishing  between  duties  and  tasks  for  their 
subordinates,  than  their  superior  Level  2  managers  are  in  distinguishing  between  duties  and  task  for 
Level  3s. Those  managers  who  do  make  a  distinction  between  duties  and  tasks  appear  to  have 
more  effective  working  relationships  with  their  subordinates. 

Indicator:  Subordinates’  goals  reflect  those  assigned  to  their  supervisors 

Most  Level  3  managers  base  subordinates’  objectives  on  their  own  performance  objectives  and  their 
superior’s  expectations  for  the  unit’s  performance,  which  is  supportive  of  the  Accountability 
element  and  of  PAA’s  ‘roll-up’  format.  Accountability  is  enhanced  when  subordinates’  performance 
objectives  support  their  superiors’  performance  objectives,  in  part  because  superiors  are  more  likely 
to  include  measurable  standards  in  those  situations.  Only  half  of  the  superiors  set  performance 
objectives  for  their  subordinates  using  milestones  in  CPME. 

Assessment:  Accountability 

Table  30,  below,  shows  the  assessment  rating  of  each  Indicator  and  the  overall  assessment  for  the 
element,  based  on  survey  findings. 


Table  30:  DRDC  MAF  Assessment,  Accountability 


Indicator 

Assessment 

Clear  accountabilities  and  responsibilities  for 
due  process  and  results 

Opportunity  for  Improvement 

Delegations  consistent  with  capabilities 

Acceptable 

Subordinates’  goals  reflect  those  assigned  to 
their  supervisors 

Strong 

OVERALL 

Acceptable 

The  TB  MAF 


speaks  of  accountabilities  for  process  (here,  ‘duties’)  and  results  (here,  ‘tasks’). 
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9  Recommendations 


Due  to  the  interdependent  nature  of  the  MAP  elements,  a  single  recommendation  can  offer 
improvement  to  multiple  elements  simultaneously;  therefore,  our  recommendations  are  not  grouped 
by  element.  Instead,  recommendations  are  presented  in  the  context  in  which  they  would  have 
greatest  effect  on  DRDC. 

Based  on  the  cyclic  development  of  the  proposed  DRDC  MAP,  supported  by  three  operational  pilot 
studies,  it  is  recommended  that: 

I.  Defence  R&D  Canada  should  adopt  the  Management  Accountability  Pramework  proposed  in 
this  report; 

1 .  This  will  require  that  DGMPRA  implement  a  DRENet-based  (or  DRENet-accessible)  on¬ 
line  survey  capability; 

2.  Note  the  utility  of  the  Public  Service  Employee  Survey  and  the  availability  of  raw  survey 
data  for  customized  analysis  in  support  of  the  DRDC  MAP; 

3.  Note  the  potential  utility  of  the  Defence  Ethics  Survey,  were  members  of  ADM  (S&T) 
group  to  be  identified  in  the  survey,  and  were  the  survey  availability  to  be  extended  to  the 
DRENet. 

II.  The  degree  of  adoption  of  those  recommendations  of  other  WBEs  of  EXPEDITION  09  that 
are  adopted  by  DRDC  should  be  assessed  through  use  of  future  DRDC  MAP  managers’ 
surveys. 

III.  Defence  R&D  should  use  the  findings  of  the  EXPEDITION  09  operational  pilots  for  the 
proposed  DRDC  MAP,  to  provide  a  starting  point  for  the  first  operational  MAP  cycle,  and 
particularly: 

1.  Design  and  Implement  a  DRDC  Succession  Plan. 

a.  Note  that  without  a  formal  succession  plan,  identification  of  high  performing  employees 
is  restricted  to  the  unit  level  and  may  both  limit  the  potential  for  an  individual  to 
progress  to  higher  levels,  and  reduce  the  pool  of  candidates  for  key  positions; 

b.  Continue  to  improve  efforts  to  increase  representation  of  designated  Employment 
Equity  groups  across  the  Agency,  particularly  within  the  management  cadre  and  its 
feeder  groups.  As  a  par  t  of  these  efforts,  encourage  specialized  professional 
development  opportunities  and  training  to  identify  and  encourage  talent  among 
designated  group  members; 

c.  Develop  and  implement  an  Agency-wide  Corporate  Knowledge  Plan.  The  number  of 
managers  encouraging  cross  training  and  knowledge  transfer  is  encouraging;  however,  a 
strategic,  organizational  plan  for  knowledge  transfer  is  necessary.  The  DRDC  mentoring 
program  is  a  positive  example  of  one  component  of  a  successful  Corporate  Knowledge 
Plan; 
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d.  It  would  be  useful  for  reeruitment  purposes  to  survey  past  student  employees  to 
determine  the  level  of  satisfaction  with  their  employment  experience,  and  also  to  see 
what  career  paths  they  have  taken  since  working  at  DRDC.  Results  of  such  a  survey 
could  help  gauge  the  corporate  image  being  promoted  to  student  population. 

2,  Increase  Support  for  and  Awareness  of  Professional  Development  Options 

a.  There  is  a  need  to  improve  awareness  and  image  of  the  DRDC  mentoring  program, 
including  its  intent,  who  is  eligible,  and  the  time  commitment  required.  A  comparison  of 
the  DRDC  and  DND  Mentoring  Program  may  provide  a  framework  for  improvement. 

A  survey  of  current  and  past  participants  to  determine  areas  of  strength  to  continue  and 
weakness  to  address  would  be  beneficial.  Weaknesses  identified  in  our  survey  include 
limited  time  to  participate  and  lack  of  vetting  and  support  for  participants; 

b.  Develop  informal  opportunities,  such  as  social  networking  events,  to  allow  managers 
with  similar  challenges  to  come  together  to  brainstorm  and  offer  mutual  support; 

c.  Offer  professional  development  and  training  in  managing  collaborative  relationships, 
with  emphasis  on  developing  communication  and  interpersonal  skills.  This  training 
should  also  highlight  the  appropriate  boundaries  that  need  to  be  maintained  between  the 
client/  partner  and  the  Agency; 

d.  Track  training  courses  and  professional  development  opportunities  undertaken  by  each 
employee  at  DRDC  in  a  central  database  (PeopleSoft)  which  moves  with  the  individual 
through  the  Agency,  and  from  which  training  and  professional  development  data  can 
easily  be  extracted  for  MAP  reporting. 

3.  Design  and  Implement  a  Corporate  Risk  Profile 

a.  Develop  an  Agency  level  corporate  risk  profile  based  on  DND  corporate  risk  profile, 
including  risk  management  roles,  responsibilities,  and  authority  limitations  for  each 
level  or  generic  role  in  DRDC; 

b.  Note  the  need  for  stronger  risk  management  competency  and  practices  in  DRDC  and 
initiate  the  inculcation  of  risk  management  practices  outside  the  TDP  program; 

c.  There  is  a  problem  with  project  documentation,  where  few  projects  formally  assign 
responsibilities.  DRDC  should  adopt  the  DND  project  responsibility  matrix  for  all 
projects,  as  a  responsibility  of  the  Project  Manager.  This  is  necessary  for  both 
Accountability  and  Risk  Management  effectiveness; 

d.  Mandatory  RM  training  (from  CSPS  or  DND)  for  all  Level  3  managers; 

e.  Adopt  the  DND  impact/  likelihood  matrix  for  project  risk  management  and  ensure  it  is 
diligently  completed  by  project  managers; 

f.  Develop  informal  opportunities,  such  as  social  networking  events,  to  allow  managers 
with  similar  challenges  to  come  together  to  brainstorm  and  offer  mutual  support; 
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g.  Create  a  ‘best  practices’  database  where  lessons  learned  from  past  experiences  can  be 
shared. 

4.  Increase  Stakeholder  Engagement 

a.  Development  and  implementation  of  Social  Impact  Assessment  Policy,  to  identify  non- 
traditional  stakeholders,  and  determine  possible  program  impacts  on  these  stakeholders; 

b.  Develop  a  Social  Impact  Checklist^''  which  can  be  used  at  different  stages  of  policy 
development  to  anticipate  possible  impacts  on  selected  stakeholders  and  society  in 
general; 

c.  Undertake  a  review  of  best  practices  in  citizen  engagement  for  military  and  scientific 
organizations  and  incorporate  findings  that  are  applicable  to  DRDC.  The  Defence 
Research  Advisory  Board  or  Defence  Science  Advisory  Board  might  offer  an  avenue 
for  this  work. 

5.  Make  Improvements  to  Performance  Management  Criteria 

a.  Make  Risk  Management  performance  a  mandatory  factor  in  managers’  performance 
appraisals.  Integrating  RM  into  Performance  Management  expectations  should  lead  to 
increased  delegation  of  Risk  Management  responsibilities  and  authorities  to 
subordinates; 

b.  Performance  objectives  with  measurable  standards  should  be  set  for  all  managers  and 
supervisors,  and  performance  against  these  used  for  performance  review.  Our  findings 
note  the  positive  effect  of  measureable  performance  standards  on  organizational 
performance;  encouraging  use  of  such  standards  will  strengthen  accountability  within 
the  management  cadre; 

c.  Note  the  need  for  greater  linkage  between  performance  objectives  and  CPME 
milestones  for  staff  who  report  to  level  3  managers; 

d.  Note  benefit  of  distinguishing  between  tasks  and  duties; 

e.  Make  use  of  PLPs  as  a  performance  review  tool.  This  supports  both  Accountability  and 
People  elements  of  the  MAP; 

f  Develop  and  implement  a  comprehensive  approach  to  setting  realistic  performance 
objectives  and  timelines  for  task  completion,  and  for  addressing  failure  to  meet 
objectives. 


See  International  Association  for  Impact  Assessment  (www.iaia.orgI  for  useful  information. 
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6.  Make  Improvements  to  Stewardship 

a.  Continue  to  ensure  delegations  are  consistent  with  capabilities; 

b.  Note  the  need  for  strengthening  of  project  management  practices,  particularly  with 
respect  to  definition  of  responsibilities  of  project  staff; 

c.  Note  the  variation  in  client  satisfaction  and  file  performance  across  DRDC  with  respect 
to  procurement  and  take  advantage  of  best  practices  where  possible; 

d.  Note  reported  deficiencies  in  protection  of  information,  whether  perceived  or  real. 
Conduct  an  investigation  to  see  if  Personnel  files  are  truly  at  risk;  if  so,  educate 
personnel  on  privacy  legislation  and  enforce  policy  to  protect  and  restrict  access  to 
information 

e.  Improve  the  processes  for  recording  and  tracking  confidential  documents  for  the 
purposes  of  auditing,  and  for  restricting  access  to  personnel  files; 

f.  Note  the  benefits  of  stronger  budgeting  and  forecasting  practices  within  the 
management  cadre; 

g.  Our  survey  findings  indicate  that  resource  and  scheduling  issues  are  most  likely  to 
contribute  to  project  failures,  thus  we  recommend  conducting  an  audit  of  milestone 
failures  to  look  at  the  impact  of  resources  and  schedule  slippage  on  outcomes. 

7.  Make  Improvements  to  Capital  Planning 

a.  Note  the  need  for  improved  capital  planning  practices,  within  the  constraints  of  limited 
Vote  5  resources; 

b.  DRDC  is  currently  in  the  process  of  a  ten  year  plan  to  renew  infrastructure  in  its  centres 
in  order  to  better  support  the  need  for  S&T  capacity  as  identified  in  the  2006  Defence 
S&T  Strategy.  As  a  part  of  this  initiative,  the  Agency  should  review  equipment  repair 
processes  and  make  necessary  improvements  to  increase  satisfaction  with  the  process 
itself  and  the  resulting  effectiveness  and  timeliness  of  repairs; 

c.  Consider  assigning  routine  maintenance  of  equipment  to  individual  positions  within  a 
unit.  This  would  not  only  support  the  stewardship  element  but  also  policy  capacity  by 
providing  valuable  information  and  feedback  for  capital  planning. 

8.  Make  Improvements  to  Policy  Process 

a.  According  to  our  survey,  the  strategic  goals  of  Canada  Advantage  appear  to  be 

adequately  represented  in  the  work  of  DRDC’s  units.  This  should  be  followed  up  with 
an  audit  for  more  objective  results;  also,  explicit  identification  of  how  each  project 
supports  the  Strategic  goals  may  prove  helpful  for  MAP  reporting  purposes; 
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b.  Examine  the  S&T  peer  review  program  for  performance  and  effectiveness.  For  example, 
is  there  follow  up  from  reviews?  Do  units  need  to  incorporate  recommendations  from 
reviews? 

c.  Note  the  need  for  partner  or  client  satisfaction  surveys  to  enhance  DRDC’s 
understanding  of  the  quality  of  its  S&T  services  to  external  clients  and  partners; 

d.  Report  results  of  client  and  partner  satisfaction  surveys  to  each  centre,  along  with  an 
overall  aggregate  report  for  the  Agency.  This  feedback  can  be  used  to  refine  elements  of 
program  and  project  delivery  as  a  part  of  a  successful  policy  process. 
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Annex  A  English  Surveys 


A.1  Survey  2 

EXPEDITION  09  SURVEY  FOR  MANAGERS 
Confidentiality 

We  thank-you  for  your  time  and  interest  in  participating  in  this  survey.  Please  note  that  all  answers 
will  remain  confidential  and  no  one  will  be  personally  identified  in  the  reporting  of  results.  Please 
take  the  time  to  read  the  following  Section  on  Consent  for  Research  Participation. 

Consent  for  Research  Participation 

1  hereby  consent  to  participate  as  a  subject  in  the  DRDC  MAP  Survey. 

1  understand  that  the  purpose  of  this  survey  is  to  support  ADM(S&T)’s  role  in  the  TB  MAP  process 
and  to  simplify  the  lives  of  managers  by  providing  clarity  in  their  roles  and  responsibilities,  and  a 
framework  with  which  to  execute  them.  The  results  of  analysis  of  this  survey  will  be  reported  to 
RDEC  in  September  2009,  and  will  be  included  in  a  DRDC  Atlantic  Contract  Report  describing  the 
development  of  the  DRDC  MAP. 

1  understand  that  my  participation  is  completely  voluntary,  and  that  1  am  free  to  withdraw  from  the 
survey  at  any  time. 

1  understand  that  the  raw  data  associated  with  this  survey  will  be  coded  in  such  a  way  that  my 
identity  will  not  be  attached  to  the  final  data  and  reports  that  are  produced. 

1  understand  that  all  reports,  whether  published  or  internal,  from  this  survey  will  be  combined  as  an 
aggregate  result,  such  that  my  identity  cannot  be  revealed  in  any  way. 


□  1  accept,  and  will  continue  with  the  survey  (Go  to  Section  A) 

□  1  do  not  accept  and  will  not  complete  the  survey  (Go  to  Section  I) 


[SCREEN  BREAK] 
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Section  A 


ALL  RESPONDENTS 

1 .  In  what  DRDC  Centre  do  you  work? 

□  Atlantic 

□  Valcartier 

□  Ottawa 

□  Toronto 

□  Suffield 

□  CORA 

□  DGMPRA 

□  Security  Science 

□  S  &  T  Operations 

□  Corporate  Services 

□  Chief  of  Staff 


2.  What  type  of  position  do  you  occupy  in  DRDC? 

□  Level  Two  (Director  General  or  Chief  of  Staff)  Go  to  Section  B 

□  Level  Three  (Deputy  Director  General,  Director,  Chief  Scientist,  Section  Head, 
Manager  or  equivalent)  Go  to  Section  Cl 

□  Level  Four  (Supervisor,  Functional  Specialist  or  equivalent)  Go  to  Section  G 

□  None  of  the  above  (LEAVE  SURVEY)  Go  to  Section  I 


SCREEN  BREAK********************************************* 
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Section  B 

LEVEL  2  MANAGERS 

3.  Does  the  organizational  chart  of  your  centre  accurately  reflect  the  centre’s  reporting  structure? 


□  Yes  (go  to  4)  □  No  (go  to  3a) 

a.  What  impact  does  the  inaccuracy  of  the  organizational  chart  have  on  your  centre? 


(textbox) 


(Proceed  to  4) 


4.  Do  you  have  confidence  that  your  subordinates  understand  their  roles? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never 

5.  Do  you  have  confidence  that  your  subordinates  understand  their  responsibilities? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never 

6.  Do  you  have  confidence  that  your  subordinates  understand  their  authority? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never 

7.  Do  you  set  performance  objectives  for  your  immediate  subordinates,  either  as  part  of  the 
annual  performance  review  process  or  as  stand-alone  objectives? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never  (go  to  Section  I) 


8.  How  do  you  set  performance  objectives  for  your  subordinates?  (Check  all  that  apply) 

□  Based  on  milestones  in  CPME  or  project  plans 

□  Based  on  my  own  superior’s  expectations  for  me 

□  Based  on  my  subordinates’  Personal  Learning  Plans 

□  Based  on  past  years’  performance 

□  Based  on  corporate  initiatives 

□  Based  on  their  work  description  or  statement  of  duties 

□  Other  (text  box) _ 
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9.  Do  your  subordinates’  performance  objectives  include  measurable  standards? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never 

10.  Do  your  subordinates’  performance  objectives  support  performance  objectives  given  to 
you  by  your  superior? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never 

Proceed  to  Section  I 
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Section  Cl 


LEVEL  3  MANAGERS 

The  following  survey  questions  relate  to  the  management  of  your  unit.  For  the  purposes  of 
this  survey,  'your  unit'  represents  the  personnel  and  resources  that  you  manage  or  supervise. 


1 1 .  How  does  your  superior  review  your  performance?  (Check  all  that  apply) 

□  Against  general  expectations 

□  Against  a  salary  administration  system  or  a  performance  management 
framework 

□  Against  the  elements  of  my  work  description  or  statement  of  duties 

□  Against  my  Personal  Learning  Plan 

□  Against  agreed  performance  objectives 

□  On  an  ad  hoc  basis 

□  Other _ (text  box) _ 


12.  Does  your  superior  set  performance  objectives  for  you,  whether  inside  a  performance 
management  agreement,  or  as  stand-alone  management  objectives? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never  (Go  to  C2) 

13.  Do  your  performance  objectives  include  measurable  standards? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never 


14.  Do  your  performance  objectives  support  your  superior’s  achievement  of  his  or  her  own 
objectives? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never  U  Uncertain/ 

Don’t  Know 


Proceed  to  Section  C2 
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Section  C2 


The  following  questions  relate  to  Risk  and  Risk  Management,  For  your  reference,  The 
Treasury  Board  of  Canada  Secretariat  defines  these  as: 

Risk:  “the  uncertainty  that  surrounds  future  events  and  outcomes.  It  is  the  expression  of  the 
likelihood  and  impact  of  an  event  with  the  potential  to  influence  the  achievement  of  an 
organization’s  objectives,” 

Risk  Management:  “a  systematic  approach  to  setting  the  best  course  of  action  under 
uncertainty  by  identifying,  assessing,  understanding,  acting  on  and  communicating  risk 
issues,” 

15.  Did  you  receive  risk  management  training  in  this  fiscal  year  or  last? 

□  Yes  □  No 

16.  How  often  are  you  consulted  for  risk  management  input  on  projects  the  outcome  of 
which  you  have  a  direct  or  related  stake? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never 

17.  Do  you  understand  your  role  in  managing  risks? 

□  Yes  □  No  □  Uncertain/ Don’t  know 

18.  Do  you  understand  your  responsibilities  for  managing  risks? 

□  Yes  □  No  □  Uncertain/ Don’t  know 

19.  Do  you  understand  the  limitations  on  your  authority  for  managing  risks? 

□  Yes  □  No  □  Uncertain/ Don’t  know 

20.  How  often  is  risk  management  discussed  at  Thrust  Advisory  Group  meetings? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never  □  Uncertain/ 

Don’t  Know 
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The  following  questions  relate  to  Project  Management.  For  your  reference,  Treasury  Board  of 
Canada  Secretariat  defines  this  as: 

Project  Management:  “the  systematic  planning,  organizing  and  control  of  allocated  resources 
to  accomplish  identified  project  objectives  and  outcomes.” 


21.  Do  you  or  your  subordinates  manage  projects? 

□  Yes  Go  to  Section  D  □  No  Go  to  Section  E 
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Section  D 


22.  Do  you  compare  reports  of  progress  on  projeets  to  organizational  plans,  for  example  by 
referenee  to  milestones  in  CPME? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never  □  N/A 


23.  Do  project  plans  explicitly  identify  speeifie  risks  in  terms  of  both  likelihood  and 
impaet? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never  □  N/A 

24.  Are  projeet  risk  assessments  revisited  during  the  delivery  of  projeets? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never  □  N/A 


25.  Where  do  you  or  your  staff  doeument  identified  projeet  risks?  (Seleet  all  that  apply) 

□  In  the  CPME  database 

□  In  project  doeumentation 

□  In  a  personal  doeument  (journal,  organizer,  PC  file) 

□  Risks  are  not  doeumented 

□  Other _ (text  box) _ 


26.  When  do  you  or  your  staff  eonsult  with  projeet  stakeholders  about  risk  issues  that  may 
affeet  them?  (Select  all  that  apply) 

□  When  planning  a  projeet 

□  During  a  seheduled  projeet  review 

□  As  risk-related  events  oecur 

□  Never 

□  Other _ (text  box) _ 


Proceed  to  Section  E 
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Section  E 


In  the  following  question,  ’duties’  are  those  day-to-day  responsihilities  that  would  he 
defined  in  your  joh  description  or  statement  of  duties,  such  as  delegated  authorities 
under  the  Financial  Administration  Act,  or  responsihilities  for  things  such  as 
occupational  health  and  safety.  ’Tasks’  are  specific  work  items  that  have  been 
assigned  hy  your  supervisor,  together  with  specific  expectations  of  performance  (such 
as  milestones). 

27.  Do  you  see  a  distinetion  between  your  ongoing  duties  and  specific  tasks  assigned  to  you 
by  your  superior? 

□  Yes  □  No  □  Uncertain/ Don’t  know 

28.  Do  you  make  progress  reports  of  work  in  your  unit  to  your  superior  during  a  formal 
program  review? 

□  Yes  □  No  □  Uncertain/ Don’t  know 


29.  Do  you  have  direct  subordinates? 

□  Yqs  Go  to  Section  FI  □  No  Go  to  Section  FI 
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Section  FI 


30.  Do  you  require  reports  of  progress  from  subordinates  in  preparation  for  or  during  a 
formal  program  review?  If  you  have  not  been  through  a  full  annual  cycle  in  your 
current  position  please  answer  N/A  unless  your  previous  position  was  also  at  Level  3;  in 
the  latter  case,  answer  based  on  your  previous  position. 

□  Yes  □  No  □  N/A 


3 1 .  Do  you  distinguish  (either  explieitly  or  implieitly)  between  ongoing  duties  and  speeifie 
tasks  that  you  assign  to  your  subordinates? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never 

32.  Do  you  set  performanee  objeetives  for  your  immediate  subordinates,  either  as  part  of  the 
annual  performanee  review  proeess  or  as  stand-alone  objeetives? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never  (Go  to 

Section  F2) 

33.  How  do  you  set  performance  objectives  for  your  subordinates?  (  Check  all  that  apply) 

□  Based  on  milestones  in  CPME  or  project  plans 

□  Based  on  my  own  superior’s  expectations  of  my  unit 

□  Based  on  my  subordinates’  Personal  Learning  Plans 

□  Based  on  past  years’  performance 

□  Based  on  corporate  initiatives 

□  Based  on  their  work  description  or  statement  of  duties 

□  Other _ (text  box) _ 


34.  Do  your  subordinates’  performance  objectives  support  your  own  performance 
objectives? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never  □  N/A 
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35.  Do  your  subordinates’  performance  objectives  include  measurable  standards? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never  □  N/A 

36.  How  often  do  you  communicate  with  your  subordinates  about  ongoing  duties? 

□  Weekly  □  Monthly  □  2-4  times  □  Annually  □  Never 

per  year 

37.  How  often  do  you  communicate  with  your  subordinates  about  assigned  tasks? 

□  Weekly  □  Monthly  □  2-4  times  □  Annually  □  Never 

per  year 

38.  In  general,  how  do  you  deal  with  situations  where  subordinates  fail  to  meet  performance 
objectives  you  have  set? 

_ (text  box) _ Proceed  to  Section  F2 
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Section  F2 


The  following  questions  relate  to  Risk  and  Risk  Management  by  your  subordinates.  For  your 
reference,  The  Treasury  Board  of  Canada  Secretariat  defines  these  as: 

Risk:  “the  uncertainty  that  surrounds  future  events  and  outcomes.  It  is  the  expression  of  the 
likelihood  and  impact  of  an  event  with  the  potential  to  influence  the  achievement  of  an 
organization’s  objectives.” 

Risk  Management:  “a  systematic  approach  to  setting  the  best  course  of  action  under 
uncertainty  by  identifying,  assessing,  understanding,  acting  on  and  communicating  risk 
issues.” 

39.  How  many  of  your  subordinates  received  risk  management  training  in  this  fiscal  year  or 
last? 

□  All  □  Some  □  None 


40.  Do  you  make  your  risk  management  priorities  and  tolerances  clear  to  your  subordinates? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never 

41.  Do  you  have  confidence  that  your  subordinates  understand  their  roles  in  managing 
risks? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never 

42.  Do  you  have  confidence  that  your  subordinates  understand  their  responsibilities  for 
managing  risks? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never 


43.  Do  you  have  confidence  that  your  subordinates  understand  the  limitations  on  their 
authority  for  managing  risks? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never 
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44.  On  average,  how  often  do  subordinates  bring  risk  issues  to  your  attention? 

□  Weekly  □  Monthly  □  2-4  times  □  Annually  □  Never 

per  year 

45.  Do  subordinates  with  responsibility  for  managing  risks  have  the  required  authority  to  do 
so  effeetively? 

□  All  □  Some  □  None 


46.  Is  risk  management  given  eonsideration  when  assessing  the  performanee  of 
subordinates? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never 

Proceed  to  Section  H 
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Section  G 

FOR  LEVEL  4  FUNCTIONAL  AUTHORITIES  OR  SUPERVISORS 

47.  Does  your  superior  set  performanee  objectives  for  you,  whether  inside  a  performance 
management  agreement,  or  as  stand-alone  management  objectives? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never  (Go  to  50) 


48.  Do  your  performance  objectives  include  measurable  standards? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never 

49.  Do  your  performance  objectives  support  your  superior’s  achievement  of  his  or  her  own 
objectives? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never  □  Uncertain/ 

Don’t  Know 


50.  Are  reports  of  your  work  progress  compared  to  organizational  plans,  for  example  by  reference  to 
milestones  in  CPME? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never  □  N/A 

5 1 .  How  does  your  superior  review  your  performance?  (Check  all  that  apply) 

□  Against  general  expectations 

□  Against  a  salary  administration  system  or  a  performance  management 
framework 

□  Against  the  elements  of  my  work  description  or  statement  of  duties 

□  Against  agreed  performance  objectives 

□  Against  my  Personal  Learning  Plan 

□  On  an  ad  hoc  basis 

□  Other _ (text  box) _ 


The  following  questions  relate  to  Risk  and  Risk  Management,  For  your  reference,  The 
Treasury  Board  of  Canada  Secretariat  defines  these  as: 

Risk:  “the  uncertainty  that  surrounds  future  events  and  outcomes.  It  is  the  expression  of  the 
likelihood  and  impact  of  an  event  with  the  potential  to  influence  the  achievement  of  an 
organization’s  objectives,” 
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Risk  Management:  “a  systematic  approach  to  setting  the  best  course  of  action  under 
uncertainty  by  identifying,  assessing,  understanding,  acting  on  and  communicating  risk 
issues.” 

52.  Do  you  understand  your  role  in  managing  risks? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never  □  N/A 

53.  Do  you  understand  your  responsibilities  for  managing  risks? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never  □  N/A 

54.  Do  you  understand  the  limitations  on  your  authority  for  managing  risks? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never  □  N/A 

55.  Are  you  consulted  for  risk  management  input  on  projects  the  outcome  of  which  you  have  a 
direct  or  related  stake? 

□  Always  □  Often  □  Sometimes  □  Rarely  □  Never  □  N/A 

In  the  following  question,  'duties'  are  those  day-to-day  responsibilities  that  would  be  defined 
in  your  job  description  or  statement  of  duties,  such  as  delegated  authorities  under  the 
Financial  Administration  Act,  or  responsibilities  for  things  such  as  occupational  health  and 
safety,  'Tasks'  are  specific  work  items  that  have  been  assigned  by  your  supervisor,  together 
with  specific  expectations  of  performance  (such  as  milestones), 

56.  Do  you  see  a  distinction  between  your  ongoing  duties  and  specific  tasks  assigned  to  you  by 
your  superior? 

□  Yes  □  No  □  Uncertain/ Don’t  Know 

Proceed  to  Section  H 
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Section  H 

FOR  LEVEL  3  AND  LEVEL  4 

For  each  of  the  following  statements,  please  indicate  your  level  of  agreement  by  circling  the 
appropriate  number. 


Strongly 

agree 

Neither 
agree  nor 
disagree 

Strongly 

disagree 

57.  I  can  disagree  with  my 
supervisor  on  work-related 
issues  without  fear  of  reprisal 

1 

2 

3 

4 

5 

58.  I  never  withhold  risk 
information  from  my 
supervisor 

1 

2 

3 

4 

5 

59.  If  I  were  to  suggest  ways  to 
improve  how  we  do  things, 
my  supervisor  would  take 
them  seriously 

1 

2 

3 

4 

5 

60.  My  supervisor  creates  an 
environment  where  it  is 
acceptable  to  not  have  all  the 
answers 

1 

2 

3 

4 

5 

61.  If  I  were  unavoidably  absent, 
a  colleague  could  step  in  to 
support  my  activities  with 
relative  ease 

1 

2 

3 

4 

5 

62.  My  supervisor  encourages 
innovation  as  an  outcome  of 
risk  taking 

1 

2 

3 

4 

5 

63 .  In  my  work  unit,  we  learn 
from  our  mistakes  and  do 
what  it  takes  to  correct  them. 

1 

2 

3 

4 

5 

64.  My  supervisor  makes  risk 
management  a  priority  in  our 
unit 

1 

2 

3 

4 

5 

65.  My  supervisor  trusts  me  to 
make  good  decisions  in  high- 
risk/  emergency  situations 

1 

2 

3 

4 

5 

66.  It  is  important  to  follow  rules 
when  dealing  with  high-risk/ 
emergency  situations 

1 

2 

3 

4 

5 
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67.  Risk  management  training  is 
a  good  use  of  my  time 

1 

2 

3 

4 

5 

68.  I  sometimes  downplay  a 
project’s  risks  in  order  to 
ensure  the  project  gets 
approval 

1 

2 

3 

4 

5 

69.  I  find  it  valuable  to  know 
how  my  peers  have  dealt 
with  risk  issues  in  their  work 

1 

2 

3 

4 

5 

Proceed  to  Section  I 
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Section  I 

ALL  RESPONDENTS 


Those  are  all  the  questions  we  have  for  you  today.  If  you  have  any  additional  comments  concerning 
this  survey,  please  leave  them  here: 

(text  box) 


Thank  you  for  completing  this  survey.  Y our  input  is  appreciated. 
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A.2  Survey  3 


EXPEDITION  09  SURVEY  FOR  MANAGERS 
Confidentiality 

We  thank-you  for  your  time  and  interest  in  participating  in  this  survey.  The  survey  should  take  no 
longer  than  30  minutes.  Please  note  that  all  answers  will  remain  confidential  and  no  one  will  be 
personally  identified  in  the  reporting  of  results.  Please  take  the  time  to  read  the  following  Section 
on  Consent  for  Research  Participation. 

Informed  Consent 

1  hereby  consent  to  participate  as  a  subject  in  the  DRDC  MAP  Survey. 

1  understand  that  the  purpose  of  this  survey  is  to  support  ADM(S&T)’s  role  in  the  TB  MAP  process 
and  to  simplify  the  lives  of  managers  by  providing  clarity  in  their  roles  and  responsibilities,  and  a 
framework  with  which  to  execute  them. 

1  understand  that  my  participation  is  completely  voluntary,  and  that  1  am  free  to  withdraw  from  the 
survey  at  any  time. 

1  understand  that  the  raw  data  associated  with  this  survey  will  be  coded  in  such  a  way  that  my 
identity  will  not  be  attached  to  the  final  data  and  reports  that  are  produced. 

1  understand  that  all  reports,  whether  published  or  internal,  from  this  survey  will  be  combined  as  an 
aggregate  result,  such  that  my  identity  cannot  be  revealed  in  any  way. 

□  1  accept,  and  will  continue  with  the  survey  (go  to  Section  A) 

□  1  do  not  accept  and  will  not  complete  the  survey  (Go  to  Section  Q) 

Hsslcslssl!******************** * C pi? pp "NJ  PR PAK 
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Section  A  -  ALL  LEVELS 


1 .  What  type  of  position  do  you  occupy  in  DRDC? 

□  Level  Two  (Director  General  or  Chief  of  Staff)  Go  to  Section  B 

□  Level  Three  (Deputy  Director  General,  Director,  Chief  Scientist,  Section  Head, 
Manager  or  equivalent)  Go  to  Section  C 

□  Level  Four  (Supervisor,  Functional  Specialist  or  equivalent)  Go  to  Section  O 

□  None  of  the  above  (LEAVE  SURVEY)  Go  to  Section  Q 
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Section  B  -  Level  2 


The  following  survey  questions  relate  to  the  management  of  designated  employment  equity 
groups,  as  identified  in  the  Employment  Equity  Act .  The  four  designated  groups  are  women, 
Aboriginal  peoples,  persons  with  disabilities,  and  visible  minorities. 

2.  To  your  knowledge,  are  designated  group  members  adequately  represented  among  feeder 
groups  for  key  positions  and  areas  in  your  centre? 

□  Yes  □  No  □  Don’t  Know 

3.  Does  your  centre  offer  professional  development  and  training  opportunities  to  identify  and 
develop  talent  among  designated  groups? 

□  Yes  □  No  □  Don’t  Know 


Go  to  Section  J 
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Section  C  -  Level  3 


4.  In  what  DRDC  Centre  do  you  work? 

□  Atlantic 

□  Valcartier 

□  Ottawa 

□  Toronto 

□  Suffield 

□  CORA 

□  DGMPRA 

□  Security  Science 

□  S  &  T  Operations 

□  Corporate  Services 

□  Chief  of  Staff 

For  the  purposes  of  this  survey,  'your  unit'  represents  the  personnel  and  resources  that  you 
manage  or  supervise. 

5.  Are  you  responsible  for  managing  a  budget  in  your  unit? 

□  Yqs  go  to  Section  D  □  No  go  to  Section  E 
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Section  D 

The  following  questions  relate  to  budget  management  in  your  unit.  For  the  purposes  of  this 
survey,  'your  unit'  represents  the  personnel  and  resources  that  you  manage  or  supervise. 


6.  In  the  last  fiscal  year,  did  your  supervisor  set  a  tolerance  for  your  end-of-year  expenditures 
relative  to  your  budget  and/or  expenditure  forecasts? 

□  Yes  (go  to  6a)  □  No  (go  to  7) 

a.  Were  your  end-of-year  expenditures  within  the  budget  or  forecast  tolerance  set  by 
your  supervisor? 


□  Yes 


□  No 


Proceed  to  question  7 

******************************************** 
7.  Does  your  unit  have  a  strategic  capital  plan  that  reflects  long-term  needs? 


□  Yes 


□  No 


1.  Does  your  unit  have  a  local  capital  equipment  plan  that  is  based  upon  your  budget  allocation? 


□  Yes  □  No 

9.  Overall,  how  would  you  rate  the  quality  of  your  working  relationship  with  your  centre’s 
financial  specialists? 

□  Poor  □  Fair  □  Good  □  Excellent  □  N/A 

The  following  questions  relate  to  procurement  activities  in  your  unit. 

10.  Is  the  number  of  procurement  specialists  in  your  centre  adequate  for  the  level  of  procurement 
activity? 

□  Never  □  Rarely  □  Sometimes  □  Often  □  Always 

1 1 .  How  many  of  the  internal  procurement  processes  in  your  organization  are  easy  to  use? 

□  None  □  Few  □  Most  □  All 

12.  Can  clients  of  the  internal  procurement  process  in  your  centre  easily  determine  progress  of 
procurement  files? 

□  Never  □  Rarely  □  Sometimes  □  Often  □  Always 
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13.  Overall,  how  would  you  rate  the  quality  of  your  working  relationship  with  your  centre’s 
procurement  specialists? 

□  Poor  □  Fair  □  Good  □  Excellent  □  N/A 


Proceed  to  Section  E 
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Section  E  -  Information  and  Asset  Management 


14.  Suppose  a  typical  unit  or  activity  under  your  authority  was  selected  for  an  audit  by  DRDC  or 
DND  auditors.  Would  the  auditor  get  a  ‘complete  picture’  by  utilizing  only  files  and  other 
records? 


Yes  go  to  Section  F2 


□  No  go  to  Section  FI 


SCREEN  RRE A K ********************************************* 
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Section  FI 


15.  Suppose  a  typical  unit  or  activity  under  your  authority  was  selected  for  an  internal  audit.  In 
order  to  have  the  ‘complete  picture’,  to  what  degree  would  the  auditor  have  to  consult  with  you 
or  your  subordinates  for  additional  input? 

□  A  significant  amount  of  input  would  be  required 

□  A  moderate  amount  of  input  would  be  required 

□  A  minor  amount  of  input  would  be  required 
Proceed  to  Section  F2 
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Section  F2 


16.  When  did  you  or  a  responsible  subordinate  last  survey  the  inventory  for  which  you  are 
accountable  (If  you  are  a  Corporate  Services  manager,  this  question  refers  to  the  inventory  in 
your  unit,  not  the  whole  organization). 

□  Within  the  last  6  months 

□  Within  the  last  7  to  12  months 

□  Within  the  last  13  to  24  months 

□  More  than  24  months  ago 

□  Never 

□  1  do  not  control  an  inventory  of  accountable  items 

17.  Do  you  or  your  subordinates  manage  projects? 

□  Yes  Go  to  Section  G  □  No  Go  to  Section  H 
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Section  G  -  Project  Management 


The  following  questions  relate  to  Project  Management,  For  your  reference,  Treasury  Board  of 
Canada  Secretariat  defines  Project  Management  as  “the  systematic  planning,  organizing  and 
control  of  allocated  resources  to  accomplish  identified  project  objectives  and  outcomes.” 

18.  For  projects  managed  within  your  unit,  does  the  project  management  documentation  include  a 
responsibility  matrix  or  some  other  formal  equivalent  assigning  responsibilities? 

□  Yes  □  No 

19.  If  a  project  managed  within  your  unit  should  fail  to  meet  planned  milestones,  in  general,  how 
likely  are  each  of  the  following  to  have  played  a  role  in  that  failure? 


Very 

Unlikely 

Somewhat 

Unlikely 

Neutral 

Somewhat 

Likely 

Very 

Likely 

Resources 

1 

2 

3 

4 

5 

Schedule 

1 

2 

3 

4 

5 

Goals 

1 

2 

3 

4 

5 

Quality  of 
milestones 

1 

2 

3 

4 

5 

20.  How  do  you  make  the  interim  or  final  project  results  available  to  stakeholders,  clients  and/or 
partners?  [choose  all  that  apply] 


□ 

Presentations 

□ 

Standard  reports 

□ 

Demonstrations 

□ 

Meeting 

□ 

Conference  papers 

□ 

Briefing  notes 

□ 

Journal  papers 

□ 

Other 

226 


DRDC  Atlantic  CR  2009-135 


2 1 .  Does  your  staff  publish  papers  or  reports  for  all  project  WBEs? 

□  Never  □  Rarely  □  Sometimes  □  Often  □  Always 

Proceed  to  Section  H 
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Section  H  -  Subordinates 


For  the  purposes  of  this  Section,  “S&T  professionals”  includes  occupational  groups  such  as 
Engineers,  Chemists  or  Computer  Scientists  who  directly  support  S&T  programs; 
occupational  groups  such  as  Technologists  are  not  included. 


22.  Do  you  have  direct  subordinates? 


□  Y es,  and  some  are  defence  scientists  or  S&T  professionals  go  to  Section  I 

□  Y es,  but  none  are  defence  scientists  or  S&T  professional  go  to  Section  J 

□  No  go  to  Section  K 
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Section  I  -  Subordinates 


For  the  purposes  of  this  Section,  S«&T  professionals  includes  occupational  groups  such  as 
Engineers,  Chemists  or  Computer  Scientists  who  directly  support  S&T  program;  occupational 
groups  such  as  Technologists  are  not  included.  For  the  purposes  of  this  question, 
collaboration  implies  face-to-face  interaction  between  your  subordinates  and  external 
stakeholders,  not  simply  participation  in  international  projects. 

23.  How  many  of  your  subordinates  are  defence  scientists  or  S&T  professionals?  Use  authorized 
staffing  level  rather  than  a  headcount  of  incumbents.  NUMBER 

b.  How  many  of  your  subordinate  defence  scientists  or  S&T  professionals  collaborate 
with  DND,  CF  or  Public  Safety  Canada  stakeholders?  NUMBER 

c.  How  many  of  your  subordinate  defence  scientists  collaborate  with  external 
Canadian  stakeholders  (including  contractors)  NUMBER 

d.  How  many  of  your  subordinate  defence  scientists  collaborate  with  foreign  partners 
under  the  auspices  of  TTCP,  NATO  or  other  international  agreements?  NUMBER 

e.  How  many  of  your  subordinate  defence  scientists  or  S&T  professionals  collaborate 
with  external  partners  on  a  fee-for-service  or  cost-recovery  basis? 

i.  Canadian  partners:  NUMBER 

ii.  Foreign  partners  :  NUMBER 
Proceed  to  Section  J 
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Section  J  -  Level  2  and  Level  3 


The  following  survey  questions  relate  to  management  of  subordinates  in  your  unit.  For  the 
purposes  of  this  survey,  'your  unit'  represents  the  personnel  and  resources  that  you  manage  or 
supervise. 

24.  Are  subordinates’  personal  learning  plans  (PLP’s)  linked  to  the  appropriate  knowledge,  skills, 
and  abilities  required  for  current  and  future  roles? 

□  Never  □  Rarely  □  Sometimes  □  Often  □  Always 

25.  Have  you  had  discussions  with  your  subordinates  regarding  the  various  possible  means  of 
passing  on  their  corporate  knowledge 

□  Yes  □  No 

26.  When  appropriate,  do  you  encourage  subordinates  to  work  in  teams  and  cross-train  to  foster 
knowledge  transfer  and  broaden  skill  sets? 

□  Never  □  Rarely  □  Sometimes  □  Often  □  Always 

27.  Have  you  considered  options  that  would  allow  you  to  bring  in  a  new  employee  while  the 
incumbent  remained  in  the  organization  in  order  to  facilitate  knowledge  transfer? 

□  Yes  go  to  27a  □  No  go  to  28 


a.  Have  you  ever  brought  in  a  new  employee  while  the  incumbent  remains  in  the 


organization  in  order  to  facilitate  knowledge  transfer? 


□  Yes  □  No 


28.  Does  your  unit  have  sufficient  FTEs  to  successfully  fulfill  your  unit's  mandate? 

□  Yes  □  No 

29.  Are  key  positions  filled  quickly? 

□  Never  □  Rarely  □  Sometimes  □  Often  □  Always 

30.  Do  new  employees  in  key  positions  perform  effectively  in  their  role  within  a  reasonable 
timeframe? 


□  Never  □  Rarely  □  Sometimes  □  Often  □  Always 
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3 1 .  Are  there  qualified  employees  in  your  unit  who  are  ready  to  compete  for  key  positions  and 
areas? 

□  Yes  □  No 

32.  Do  you  identify  'high-performing'  employees  for  succession  planning? 


□  Yes  □  No 


Proceed  to  Section  K 
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Section  K  -  Level  2,  Level  3 

The  following  survey  questions  relate  to  the  peer  review  process. 

33.  Is  your  unit  subject  to  S&T  peer  reviews? 

□  Yqs  go  to  Section  L  □  No  go  to  Section  N  □  Don’t  know  goto 

Section  N 
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Section  L  -  Level  2,  Level  3 


34.  When  was  the  last  peer  review  of  an  S&T  activity  in  your  unit? 

□  One  year  ago 

□  Two  years  ago 

□  Three  Y ears  ago 

□  Four  years  ago 

□  Five  years  ago 

□  More  than  5  years  ago 

□  Never 

□  Unknown  Go  to  section  N 

Others  Proceed  to  Section  L2 


DRDC  Atlantic  CR  2009-135 


233 


Section  L2  -  Level  2,  Level  3 

35.  Overall,  what  was  the  outcome  of  your  last  peer  review? 

□  Very  L  Somewhat  □  Neutral  □  Somewhat  □  Very  □  Don’t  Know 

Negative  Negative  Positive  Positive  Go  to  section  N 


Go  to  Section  M— 


SCREEN  BREAK********************************************* 
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Section  M  -  Level  2,  Level  3 


36.  Overall,  did  your  last  peer  review  offer  valuable  feedback? 
□  Not  Valuable  □  Neutral  □  Valuable 


37.  In  general,  how  much  impact  does  peer  review  have  on  your  unit’s  capabilities  and 
competencies? 

□  No  □  Some  □  Significant 

Impact  Impact  Impact 

38.  In  general,  what  kind  of  impact  does  peer  review  have  on  your  unit’s  capabilities  and 
competencies? 

□  Negative  □  Neutral/  □  Positive 

Impact  No  Impact  Impact 


39.  In  general,  how  much  impact  does  peer  review  have  on  the  outcomes  of  your  projects? 

□  No  □  Some  □  Significant 

Impact  Impact  Impact 

40.  In  general,  what  kind  of  impact  does  peer  review  have  on  the  outcomes  of  your  projects? 

□  Negative  □  Neutral/  □  Positive 

Impact  No  Impact  Impact 


Proceed  to  Section  N 
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Section  N  -  Level  2,  Level  3 

The  following  questions  relate  to  working  and  collaborative  relationships  with  partners, 
clients,  and  contractors.  For  the  purposes  of  this  survey,  collaborative  means  ‘to  work  jointly 
on  an  initiative  to  achieve  a  common  purpose,  while  sharing  accountabilities’. 

41.  How  would  you  rate  your  unit’s  collaborative  working  relationship  with  the  private  sector 
S&T  community? 

□  Poor  □  Fair  □  Good  □  Excellent  □  N/A 

42.  How  would  you  rate  your  unit’s  collaborative  working  relationship  with  the  academic 
community? 

□  Poor  □  Fair  □  Good  □  Excellent  □  N/A 

43.  Were  you  informed  of  the  results  of  the  most  recent  partner/client  satisfaction  survey  for 
your  unit? 

□  Yes  □  No  □  N/A 

44.  How  often  do  you  receive  feedback  directly  from  clients  or  partners  after  S&T  products 
enter  service? 

□  Never  □  Rarely  □  Sometimes  □  Often  □  Always  □  N/A 

45.  Does  feedback  from  partners  and  clients  (includes  both  direct  and  survey  feedback) 
influence  the  design  or  delivery  of  future  projects? 

□  Never  □  Rarely  □  Sometimes  □  Often  □  Always  □  N/A 

46.  How  often  do  you  collaborate  with  others  when  developing  policies  or  programs? 

□  Rarely  □  Sometimes  □  Often  □  Always 

- Go  to  question  46a - 


□  Never 

Level  2-  Go  to  Section  P 
Level  3-  Go  to  Section  O 


b.  In  general,  how  would  you  assess  the  quality  of  your  collaborative  relationships? 

□  Poor  □  Fair  □  Good  □  Excellent 

c.  To  what  extent  did  your  most  recent  collaboration  achieve  its  intended  goals  or 
requirements? 
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□  Achieved  □  Achieved  some  □  Achieved  most  □  Achieved  all 

none 

Level  2-  Proceed  to  Section  P 
Level  3-  Proceed  to  Section  O 
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Section  O  -  Level  3  and  Level  4 

The  following  questions  relate  to  maintenance  of  equipment  utilized  in  your  unit.  For  the 
purposes  of  this  survey,  'your  unit'  represents  the  personnel  and  resources  that  you  manage  or 
supervise, 

47.  Is  daily/routine  maintenance  of  program-related  equipment  (e.g.  calibration,  etc)  formally 
assigned  to  a  designed  individual  in  your  unit? 

□  Never  □  Rarely  □  Sometimes  □  Often  □  Always  □  N/A 

48.  Is  daily/routine  maintenance  of  infrastructure-related  equipment  (e.g.  toner  replacement,  etc) 
formally  assigned  to  a  designed  individual  in  your  unit? 

□  Never  □  Rarely  □  Sometimes  □  Often  □  Always  □  N/A 

49.  When  equipment  needs  special  maintenance  or  repair,  is  the  process  used  for  the  request 
easy  to  use? 

□  Never  □  Rarely  □  Sometimes  □  Often  □  Always 

50.  Are  equipment  repair  requests  addressed  in  a  timely  and  effective  manner? 

□  Never  □  Rarely  □  Sometimes  □  Often  □  Always 


The  following  questions  relate  your  experiences  in  your  work  unit, 

5 1 .  How  important  is  it  to  your  job  performance  to  stay  abreast  of  new  ideas,  methods  and 
technology  in  your  field? 

□ 

Very  □  Somewhat  □  Somewhat  □  Very 

Unimportant  Unimportant  Important  Important 

52.  Are  you  satisfied  with  the  resources  and/or  opportunities  provided  by  DRDC  to  stay  abreast 
of  new  ideas,  methods  and  technology  in  your  field? 

□  Very  □  Somewhat  □  Neither  □  Somewhat  □  Very  Satisfied 

Dissatisfied  Dissatisfied  Satisfied  nor  Satisfied 

Dissatisfied 

53.  Has  your  unit  worked  with  students  in  your  unit  in  the  past?  (FSWEP,  co-op,  internships, 
etc.) 

□  Yes  go  to  Section  02  □  No  go  to  Section  P 
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Section  02  -  Level  3  and  Level  4 

54.  How  would  you  rate  your  unit’s  experience  working  with  students? 

□  Poor  □  Fair  □  Good  □  Excellent 

Proceed  to  Section  P 
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Section  P  -  ALL 


The  following  question  relates  to  broad  strategic  goals.  For  the  purposes  of  this  survey,  'your 
unit'  represents  the  personnel  and  resources  that  you  manage  or  supervise. 


55.  In  general,  how  frequently  are  the  following  broad  goals  an  explicit  factor  in  your  unit’s 
work? 


Never 

Rarely 

Sometimes 

Often 

Always 

Workforce 

development 

1 

2 

3 

4 

5 

Maintenance  of 
national  scientific  and 
technical  infrastructure 
and  capacity 

1 

2 

3 

4 

5 

Expansion  of  the  body 
of  knowledge 

1 

2 

3 

4 

5 

Commercialization 

1 

2 

3 

4 

5 

Mission  directed  needs 
for  R&D  results 

1 

2 

3 

4 

5 

The  following  questions  relate  to  information  management  in  your  centre. 

56.  In  your  centre,  is  access  to  personnel  files  restricted  to  human  resources  staff  only? 

□  Yes  □  No  □  Don’t  know 

57.  In  your  centre,  is  classified  correspondence  tracked?  (e.g.  unique  user  ID,  document  log, 
etc) 

□  Yes  □  No  □  Don’t  know 


240 


DRDC  Atlantic  CR  2009-135 


58.  In  your  centre,  is  classified  correspondence  filed  securely? 

□  Yes  □  No 

The  following  questions  relate  to  the  DRDC  Mentoring  Program, 

59.  Have  you  been  involved  in  the  DRDC  Mentoring  Program? 

□  Yes  go  to  59a  □  No  go  to  59c 

a.  In  what  capacity  were  you  involved?  Check  all  that  apply 

□  Mentor 

□  Mentee 

□  Member  of  mentoring  program  team 

b.  How  would  you  rate  your  satisfaction  with  the  DRDC  Mentoring  Program? 

□  Poor  □  Fair  □  Good  □  Excellent 

Go  to  question  60 

c.  If  no,  what  is  your  impression  of  the  DRDC  Mentoring  Program? 

□  Positive  □  Negative  □  No  Opinion  □  Unaware  of  Program 

Go  to  question  60 
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60.  Are  you  interested  in  participating  in  the  DRDC  Mentoring  Program  in  the  future? 
□  Yes  go  to  60a  □  No  go  to  61  □  Unsure  go  to  61 

a.  In  what  capacity  would  you  be  interested  in  participating  in  the  Mentoring 
Program? 

□  Mentor 

□  Mentee 

□  Member  of  mentoring  program  team 


Proceed  to  question  61 


61.  Please  note  any  comments  on  the  DRDC  Mentoring  Program  here: 
(textbox) 


Proceed  to  Section  Q 
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Section  Q 

Those  are  all  the  questions  we  have  for  you  today.  If  you  have  any  additional  comments  concerning 
this  survey,  please  leave  them  here: 

(text  box) 


Thank  you  for  completing  this  survey.  Your  input  is  appreciated. 
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This  page  intentionally  left  blank. 
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Annex  B  French  Surveys 


B.1  Sondage  2 

SONDAGE  A  L’INTENTION  DES  GESTIONNAIRES  POUR  EXPEDITION  09 
Anonymat 

Nous  remercions  de  I’interet  que  vous  avez  manifeste  pour  cette  enquete  et  du  temps  que 
vous  voudrez  bien  y  consacrer.  Veuillez  no  ter  que  toutes  les  reponses  resteront 
eonfidentielles  et  que  personne  ne  sera  identifie  personnellement  dans  la  presentation  des 
resultats.  Veuilliez  lire  a  seetion  suivante  au  sujet  de  eonsentement. 

Consentement 

Je  eonsens,  par  la  presente,  a  partieiper  a  1’ enquete  sur  le  CRG  pour  RDDC. 

J’ai  bien  eompris  que  la  fonetion  de  eette  enquete  est  de  soutenir  le  role  du  SMA(S  et  T) 
dans  le  processus  du  CRG  du  Conseil  du  Tresor  et  de  simplifier  la  vie  des  gestionnaires  en 
leur  fournissant  des  roles  et  des  responsabilites  clairs  ainsi  qu’un  outil  qui  les  aidera  a 
mettre  ceux-ci  en  oeuvre.  On  donnera  les  resultats  du  sondage  pemierement  au  comite 
executif  de  rddc  en  septembre  2009  et  deuxiemement  dans  un  rapport  de  contracteur  de  rddc 
Atlantique. 

J’ai  bien  eompris  que  ma  participation  a  cette  enquete  se  fait  a  titre  purement  volontaire  et 
que  je  suis  libre  de  me  retirer  a  tout  moment. 

J’ai  bien  eompris  que  les  donnees  brutes  de  cette  enquete  seront  encodees  de  fa9on  a  ce  que 
mon  identite  ne  soit  pas  associee  aux  donnees  de  synthese  et  aux  rapports  qui  seront 
produits  a  partir  de  ces  donnees. 

J’ai  bien  eompris  que  tous  les  rapports,  qu’ils  soient  publics  ou  internes,  produits  a  partir  de 
cette  enquete  seront  des  documents  de  synthese  qui  ne  reveleront  pas  mon  identite  de 
quelque  fa9on  que  ce  soit 

□  Je  eonsens  a  partieiper  (continue  with  survey) 

□  Je  ne  eonsens  pas  a  partieiper  (Go  to  Section  I) 


[SCREEN  BREAK] 
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Section  A 


A  TOUS  LES  REPONDANTS 

1 .  A  quel  centre  de  RDDC  travaillez-vous? 

□  Atlantique 

□  Valcartier 

□  Ottawa 

□  Toronto 

□  Suffield 

□  CARO 

□  DGRAPM 

□  Sciences  pour  la  securite 

□  Operations  S  &  T 

□  Services  generaux 

□  Chef  d’etat-major 

2.  Quel  type  de  poste  occupez-vous  a  RDDC? 

□  Niveau  deux  (directeur  general  ou  chef  d’etat-major)  Go  to  Section  B 

□  Niveau  trois  (directeur  general  adjoint,  directeur,  scientifique  en  chef,  chef  de 
section,  gestionnaire  ou  Tequivalent)  Go  to  Section  Cl 

□  Niveau  quatre  (superviseur,  expert  fonctionnel  ou  Tequivalent) Go  to  Section  G 

□  Aucun  de  ce  qui  precede  (QUITTER  LE  SONDAGE)  Go  to  Section  I 


SCREEN  BREAK********************************************* 
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Section  B 


GESTIONNAIRES  DE  NIVEAU  2 


3.  L’ organigramme  de  votre  centre  temoigne-t-il  fidelement  de  la  structure  hierarchique  du 

centre? 

□  Oui  (go  to  4)  □  Non  (go  to  3a) 

Hs****************§(;;;]^££]Sj  gj^^j^********************************************* 

a.  Quel  effet  I’inexactitude  de  rorganigramme  exerce-t-elle  sur  votre  centre? 


(textbox) 


_ (Proceed  to  4) 

SCREEN  BREAK********************************************* 


4.  Etes-vous  persuade  que  vos  subordonnes  comprennent  leurs  roles? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais 

5.  Etes-vous  persuade  que  vos  subordonnes  eomprennent  leurs  responsabilites? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais 

6.  Etes-vous  persuade  que  vos  subordonnes  eomprennent  leurs  pouvoirs? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais 

7.  Eixez-vous  des  objectifs  de  rendement  a  I’intention  de  vos  subordonnes  immediats, 
soit  dans  le  eadre  du  proeessus  annuel  d’examen  du  rendement,  soit  en  tant 
qu’objeetifs  independants? 


□  Toujours  □  Souvent  □  Parfois 


□  Rarement  □  Jamais  (Go  to 
Section  I) 


SCREEN  rrf.AK ********************************************* 
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8.  Comment  etablissez-vous  les  objectifs  de  rendement  de  vos  subordonnes?  (Cocher  tons  les 
points  pertinents) 

□  d’apres  les  jalons  du  SCPG  ou  les  plans  des  projets 

□  en  fonction  des  attentes  de  mon  superieur  a  mon  egard 

□  en  fonction  de  leur  plan  d’apprentissage  personnel 

□  en  fonction  du  rendement  des  annees  anterieures 

□  selon  les  initiatives  de  1’ organisation 

□  en  fonction  de  leur  description  de  travail  ou  de  I’enonce  de  mes  fonctions 

□  autre 


9.  Les  objectifs  de  rendement  de  vos  subordonnes  integrent-ils  des  normes  mesurables? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais 

10.  Les  objectifs  de  rendement  de  vos  subordonnes  soutiennent-ils  vos  propres  objectifs  de 
rendement? 


□  Toujours  □  Souvent  □  Parfois 


□  Rarement  □  Jamais 


Proceed  to  Section  I 
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Section  Cl 


GESTIONNAIRES  DE  NIVEAU  3 

Les  questions  qui  suivent  se  rapportent  a  la  gestion  de  votre  unite.  Pour  les  besoins  du 
sondage,  «  votre  unite  »  designe  le  personnel  et  les  ressources  que  vous  gerez  on  supervisez, 

1 1 .  Comment  votre  superieur  evalue-t-il  votre  rendement?  (Coeher  tons  les  points 
pertinents) 

□  par  rapport  aux  attentes  generales 

□  en  fonetion  d’un  systeme  d’ administration  de  la  remuneration  ou  d’un  cadre  de 
gestion  du  rendement 

□  par  rapport  aux  elements  de  ma  description  de  travail  ou  mon  enonce  des 
fonctions 

□  en  fonetion  de  mon  plan  d’apprentissage  personnel 

□  par  rapport  aux  objectifs  convenus 

□  de  fa9on  aleatoire 

□  autre 


12.  Votre  superieur  etablit-il  des  objectifs  de  rendement  qui  integrent  des  normes 
mesurables  pour  vous,  soil  dans  le  contexte  d’une  entente  de  gestion  du  rendement,  soil 
en  tant  qu’objectifs  independants  de  gestion? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais  (Go  to  C2) 

13.  Vos  propres  objectifs  de  rendement  integrent-ils  des  normes  mesurables? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais 

14.  Vos  objectifs  de  rendement  soutiennent-ils  la  realisation  par  votre  superieur  de  ses 
propres  objectifs? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais  □  Incertain/ 

Ne  sais  pas 


Proceed  to  Section  C2 
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Section  C2 


Les  questions  suivantes  se  rapportent  an  risque  et  a  la  gestion  du  risque,  A  titre  de 
reference,  voici  les  definitions  du  Secretariat  du  Conseil  du  Tresor  a  ces  sujets  : 

Le  risque  se  rapporte  a  I’incertitude  qui  entoure  des  evenements  et  des  resultats 
futurs,  II  est  I’expression  de  la  probabilite  et  de  I’incidence  d’un  evenement  susceptible 
d’influencer  I’atteinte  des  objectifs  de  I’organisation. 

La  gestion  du  risque  est  une  approcbe  systematique  servant  a  determiner  la  meilleure 
voie  a  prendre  en  cas  d’incertitude  en  identifiant,  en  evaluant,  en  comprenant,  en 
communiquant  les  questions  liees  aux  risques  et  en  prenant  des  mesures  a  leur  egard. 

15.  Avez-vous  re9u  une  formation  en  gestion  des  risques  au  cours  de  eette  annee  financiere 
ou  de  la  derniere  annee  financiere? 

□  Oui  □  Non 

16.  A  quelle  frequence  est-ce  qu’on  vous  consulte  relativement  a  la  gestion  des  risques  de 
projets  dont  les  resultats  vous  sont  directement  ou  indirectement  imputables? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais 

17.  Comprenez-vous  vos  roles  en  fait  de  gestion  des  risques? 

□  Oui  □  Non  □  Incertain/  Ne  sais  pas 

18.  Comprenez-vous  vos  responsabilites  en  fait  de  gestion  des  risques? 

□  Oui  □  Non  □  Incertain/  Ne  sais  pas 

19.  Comprenez-vous  les  limites  de  vos  pouvoirs  en  fait  de  gestion  des  risques? 

□  Oui  □  Non  □  Incertain/  Ne  sais  pas 

20.  A  quelle  frequence  la  gestion  des  risques  fait-elle  I’objet  de  discussions  aux  reunions 
du  Groupe  consultatif  sur  les  vecteurs? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais  □  Incertain/ 

Ne  sais  pas 
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Les  questions  suivantes  se  rapportent  a  la  gestion  de  projet,  A  titre  de  reference,  voici 
la  definition  du  Conseil  du  Tresor  a  ce  sujet : 

La  gestion  de  projet  designe  la  planification,  I’organisation  et  la  surveillance 
systematiques  des  ressources  attribuees  pour  atteindre  certains  objectifs  et  resultats 
des  projets, 

21.  Est-ce  que  vous-meme  ou  vos  subordonnes  gerez  des  projets? 

□  Oui  Go  to  Section  D  □  Non  Go  to  Section  E 
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Section  D 


22.  Les  comptes  rendus  de  ravancement  des  projets  se  font-ils  comparativement  aux  plans 
organisationnels,  par  exemple  par  rapport  aux  jalons  du  SCPG? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais  □  S.O. 

23.  Les  plans  des  projets  sont-ils  explieites  pour  ce  qui  est  de  I’identifieation  et  de 
1’ evaluation  des  risques? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais  □  S.O. 

24.  Les  evaluations  des  risques  des  projets  sont-elles  reexaminees  au  moment  de  la 
livraison  des  projets? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais  □  S.O. 

25.  Par  quel  moyen  vous-meme  ou  votre  personnel  consignez-vous  les  risques  reeonnus 
des  projets?  (Coeher  tous  les  points  pertinents) 

□  Dans  la  base  de  donnees  du  SCPG 

□  Dans  la  doeumentation  eoneernant  le  projet 

□  Dans  un  doeument  personnel  (journal  de  bord,  organiseur,  dossier  informatique) 

□  Les  risques  ne  sont  pas  doeumentes 

□  Autre _ (text  box) _ 

26.  A  quel  moment  vous-meme  ou  votre  personnel  eonsultez-vous  les  intervenants  des 
projets  au  sujet  des  risques  pouvant  les  toueher? 

□  Lors  de  la  planification  d’un  projet 

□  Lors  des  etapes  prevues  d’examen  du  projet 

□  A  mesure  que  surviennent  les  evenements  lies  aux  risques 

□  Jamais 

□  Autre  oeeasion 


Proceed  to  Section  E 
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Section  E 


Dans  le  contexte  des  questions  ci-apres,  le  terme  «  functions  »  designe  les 
responsabilites  quotidiennes  qui  seraient  enoncees  dans  votre  description  de  travail  on 
enonce  des  functions,  tels  les  pouvoirs  delegues  en  vertu  de  la  Loi  sur  la  gestion  des 
finances  publiques,  on  les  responsabilites  a  I’egard  entre  autres  de  la  sante  et  securite 
an  travail.  Les  «  tacbes  »  s’entendent  des  elements  de  travail  que  vous  confie  votre 
superviseur,  ainsi  que  des  attentes  precises  de  rendement  (comme  les  jalons), 

27.  Percevez-vous  une  distinction  entre  vos  fonetions  et  les  taches  que  vous  attribue  votre 
superieur? 

□  Oui  □  Non  □  Incertain/  Ne  sais  pas 

28.  Presentez-vous  des  rapports  d’avaneement  sur  le  travail  de  votre  unite  a  votre  superieur 
dans  le  contexte  d’un  examen  formel  des  programmes? 

□  Oui  □  Non  □  Incertain/  Ne  sais  pas 

29.  Avez-vous  des  subordonnes  qui  relevent  direetement  de  vous? 

□  Yqs  Go  to  Section  FI  □  No  Go  to  Section  H 


Section  FI 

30.  Exigez-vous  de  vos  subordonnes  des  rapports  d’avaneement  en  prevision  ou  dans  le 
cadre  d’un  examen  formel  des  programmes?  Si  vous  n  'avez  pas  encore  accompli  un 
cycle  annuel  complet  dans  votre poste  actuel,  veuillez  repondre  d’apres  votre 
experience  anterieure  au  mime  niveau  professionnel  a  RDDC,  ou  selon  ce  que  vous 
comptez  faire. 

□  Oui  □  Non  □  S.O. 


3 1 .  Faites-vous  une  distinction  (explicitement  ou  implicitement)  entre  les  fonetions  et  les 
taehes  que  vous  confiez  a  vos  subordonnes? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais 


DRDC  Atlantic  CR  2009-135 


253 


32.  Fixez-vous  des  objectifs  de  rendement  a  I’intention  de  vos  subordonnes  immediats,  soit 
dans  le  cadre  du  processus  annuel  d’examen  du  rendement,  soit  en  tant  qu’ objectifs 
independants? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais  (Go  to 

Section  F2) 

33.  Comment  etablissez-vous  les  objectifs  de  rendement  de  vos  subordonnes?  (Cocher  tons  les 
points  pertinents) 

□  d’apres  les  jalons  du  SCPG  ou  les  plans  des  projets 

□  en  fonction  des  attentes  de  mon  superieur  a  I’egard  de  mon  unite 

□  en  fonction  de  leur  plan  d’apprentissage  personnel 

□  en  fonction  du  rendement  des  annees  anterieures 

□  selon  les  initiatives  de  P  organisation 

□  en  fonction  de  leur  description  de  travail  ou  de  I’enonce  de  mes  fonctions 

□  autre _ 

34.  Les  objectifs  de  rendement  de  vos  subordonnes  soutiennent-ils  vos  propres  objectifs  de 
rendement? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais  □  S.O. 

35.  Les  objectifs  de  rendement  de  vos  subordonnes  integrent-ils  des  normes  mesurables? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais  □  S.O. 

36.  A  quelle  frequence  communiquez-vous  avec  vos  subordonnes  au  sujet  des  fonctions 
executees? 

□  Une  fois  □  Une  fois  □  2-4  fois  par  □  Une  fois  □  Jamais 

par  par  mois  annee  par  annee 

semaine 

37.  A  quelle  frequence  communiquez-vous  avec  vos  subordonnes  au  sujet  des  taches  qui 
sont  attribuees? 

□  Une  fois  □  Une  fois  □  2-4  fois  par  □  Une  fois  □  Jamais 

par  par  mois  annee  par  annee 

semaine 
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38.  En  regie  generale,  eomment  reagissez-vous  lorsque  des  subordonnes  n’atteignent  pas 
les  objeetifs  que  vous  fixez? 

(text  box) 

Proceed  to  Section  F2 
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Section  F2 


Les  questions  suivantes  se  rapportent  an  Risque  et  a  la  gestion  du  risque  par  vos 
subordonnes.  A  titre  de  reference,  voici  les  definitions  du  Secretariat  du  Conseil  du 
Tresor  a  ces  sujets 

Le  risque  se  rapporte  a  I’incertitude  qui  entoure  des  evenements  et  des  resultats 
futurs,  II  est  I’expression  de  la  probabilite  et  de  I’incidence  d’un  evenement  susceptible 
d’influencer  I’atteinte  des  objectifs  de  I’organisation. 

La  gestion  du  risque  est  une  approcbe  systematique  servant  a  determiner  la  meilleure 
voie  a  prendre  en  cas  d’incertitude  en  identifiant,  en  evaluant,  en  comprenant,  en 
communiquant  les  questions  liees  aux  risques  et  en  prenant  des  mesures  a  leur  egard. 


39.  Combien  de  vos  subordonnes  ont  re9u  une  formation  en  gestion  des  risques  au  cours  de 
cette  annee  financiere  ou  de  la  demiere  annee  financiere? 

□  Tous  □  Certains  □  Aucun 

40.  Presentez-vous  clairement  a  vos  subordonnes  vos  priorites  et  vos  seuils  de  tolerance  en 
matiere  de  gestion  des  risques? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais 

41.  Etes-vous  persuade  que  vos  subordonnes  comprennent  leurs  roles  en  matiere  de  gestion 
des  risques? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais 

42.  Etes-vous  persuade  que  vos  subordonnes  comprennent  leurs  responsabilites  en  matiere 
de  gestion  des  risques? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais 

43.  Etes-vous  persuade  que  vos  subordonnes  comprennent  les  limites  de  leurs  pouvoirs  en 
matiere  de  gestion  des  risques? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais 

44.  En  moyenne,  a  quelle  frequence  vos  subordonnes  portent-ils  des  problemes  [en  matiere 
de  gestion  des  risques]  a  votre  attention? 

□  Une  fois  par  □  Une  fois  □  2-4  fois  par  □  Une  fois  par  □  Jamais 

semaine  par  mois  annee  annee 
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45.  Les  subordonnes  responsables  de  la  gestion  des  risques  detiennent-ils  les  pouvoirs  leur 
permettant  de  s’acquitter  de  ee  role  avee  effieaeite? 

□  Tous  □  Certains  □  Aucun 


46.  La  gestion  des  risques  est-elle  prise  en  eonsideration  lors  de  revaluation  du  rendement 
de  VOS  subordonnes? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais 

Proceed  to  Section  H 
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Section  G 

Responsables  fonctionnels  on  superviseurs  de  niveau  4 

47.  Votre  superieur  etablit-il  des  objectifs  de  rendement  qui  integrent  des  normes  mesurables  a 
votre  egard,  sous  forme  d’ entente  de  gestion  du  rendement  ou  en  tant  qu’ objectifs  de  gestion 
independants? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais  (Go  to  50) 


SCREEN  RREAR ********************************************* 


48.  Vos  propres  objectifs  de  rendement  integrent-ils  des  normes  mesurables? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais 

49.  Vos  objectifs  de  rendement  soutiennent-ils  la  realisation  des  objectifs  de  votre  superieur? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais  □  Incertain/ Ne 

sais  pas 


50.  Ees  comptes  rendus  de  vos  progres  se  font-ils  comparativement  aux  plans  organisationnels,  par 
exemple  par  rapport  aux  jalons  du  SCPG? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais  □  S.O. 

5 1 .  Comment  votre  superieur  evalue-t-il  votre  rendement?  (Cocher  tons  les  points 
pertinents) 

□  par  rapport  aux  attentes  generales 

□  en  fonction  d’un  systeme  d’ administration  de  la  remuneration  ou  d’un  cadre  de 
gestion  du  rendement 

□  par  rapport  aux  elements  de  ma  description  de  travail  ou  mon  enonce  des 
fonctions 

□  par  rapport  aux  objectifs  convenus 

□  en  fonction  de  mon  plan  d’apprentissage  personnel 

□  de  fa9on  aleatoire 

□  autre _ 

Les  questions  suivantes  se  rapportent  au  risque  et  a  la  gestion  du  risque,  A  titre  de 
reference,  voici  les  definitions  du  Secretariat  du  Conseil  du  Tresor  a  ces  sujets  : 
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Le  risque  se  rapporte  a  I’incertitude  qui  entoure  des  evenements  et  des  resultats 
futurs.  II  est  I’expression  de  la  probabilite  et  de  I’incidence  d’un  evenement  susceptible 
d’influencer  I’atteinte  des  objectifs  de  I’organisation. 

La  gestion  du  risque  est  une  approche  systematique  servant  a  determiner  la  meilleure 
voie  a  prendre  en  cas  d’incertitude  en  identiflant,  en  evaluant,  en  comprenant,  en 
communiquant  les  questions  liees  aux  risques  et  en  prenant  des  mesures  a  leur  egard. 

52.  Comprenez-vous  vos  roles  en  matiere  de  gestion  des  risques? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais  □  S.O. 

53.  Comprenez-vous  vos  responsabilites  en  matiere  de  gestion  des  risques? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais  □  S.O. 

54.  Comprenez-vous  les  limites  de  vos  pouvoirs  en  matiere  de  gestion  des  risques? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais  □  S.O. 

55.  Est-ce  qu’on  vous  consulte  relativement  a  la  gestion  des  risques  de  projets  dont  les  resultats 
vous  sont  directement  ou  indirectement  imputables? 

□  Toujours  □  Souvent  □  Parfois  □  Rarement  □  Jamais  □  S.O. 

Dans  le  contexte  de  la  question  ci-apres,  le  terme  «  fonctions  »  designe  les  responsabilites 
quotidiennes  qui  seraient  enoncees  dans  votre  description  de  travail  ou  enonce  des  fonctions, 
tels  les  pouvoirs  delegues  en  vertu  de  la  Loi  sur  la  gestion  des  finances  publiques,  ou  les 
responsabilites  a  I’egard  entre  autres  de  la  saute  et  securite  an  travail.  Les  «  taches  » 
s’entendent  des  elements  de  travail  que  vous  confie  votre  superviseur,  ainsi  que  des  attentes 
precises  de  rendement  (comme  les  jalons). 

56.  Percevez-vous  une  distinction  entre  vos  fonctions  et  les  taches  que  vous  attribue  votre 
superieur? 

□  Oui  □  Non  □  Incertain/  Ne  sais  pas 

Proceed  to  Section  H 
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Section  H 

RESPONSABLES  DE  NIVEAU  3  OU  4 

Pour  chacun  des  enonces  qui  suivent,  veuillez  indiquer  votre  degre  d’ accord  en  encerclant  le 
chiffre  approprie. 


Fortcmcnt 

d’accord 

Ni  d’accord 
ni  cn 

desaccord 

Fortement 

en 

desaccord 

57.  Je  peux  etre  en  desaccord 
avec  mon  superviseur  en  ce 
qui  conceme  le  travail  sans 
crainte  de  represailles. 

1 

2 

3 

4 

5 

58.  Je  n’epargne  jamais  mon 
superviseur  de 
renseignements  concemant 
les  risques. 

1 

2 

3 

4 

5 

59.  Si  je  proposais  des  moyens 
d’ameliorer  nos  fapons  de 
proceder,  mon  superviseur 
les  prendrait  an  serieux. 

1 

2 

3 

4 

5 

60.  Mon  superviseur  cree  un 
climat  dans  lequel  il  est 
acceptable  de  ne  pas  avoir 
toutes  les  reponses. 

1 

2 

3 

4 

5 

61.  Si  je  devais  etre 

inevitablement  absent,  un 
collegue  pourrait  me 
remplacer  assez  aisement 
dans  mes  activites. 

1 

2 

3 

4 

5 

62.  Mon  superviseur  encourage 
I’innovation  en  taut  que 
resultat  de  la  prise  de 
risques. 

1 

2 

3 

4 

5 

63 .  Dans  mon  unite  de  travail, 
nous  apprenons  de  nos 
erreurs  et  nous  prenons  les 
mesures  qui  s’imposent  pour 
les  corriger. 

1 

2 

3 

4 

5 

64.  Mon  superviseur  fait  de  la 
gestion  du  risque  une  priorite 
de  notre  unite. 

1 

2 

3 

4 

5 

65.  Mon  superviseur  me  fait 
confiance  pour  prendre  de 
bonnes  decisions  dans  des 
situations  a  haut 
risque/d’urgence. 

1 

2 

3 

4 

5 
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66.  Le  respect  des  regies  est 
important  lorsqu’on 
traite  des  situations  a 
haut  risque/d’urgence. 

1  2  3  4  5 

67.  J’ai  trouve  le  temps 
passe  a  la  formation  sur 
la  gestion  du  risque  bien 
utile. 

1  2  3  4  5 

68.  Je  minimise  parfois  les 
risques  lies  a  un  projet 
pour  m’assurer  que  le 
projet  est  approuve. 

1  2  3  4  5 

69.  Je  trouve  tres  utile  le  fait 
que  mes  pairs  aient  eu  a 
traiter  des  questions  de 
risque  dans  leur  travail. 

1  2  3  4  5 

Proceed  to  Section  I 
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Section  I 


Si  vous  souhaitez  faire  d’autres  commentaires  au  sujet  de  ce  sondage-ci,  notez  vos 
commentaires  ci-dessous. 

(text  box) 


Merci  d’avoir  pris  le  temps  de  remplir  le  questionnaire.  Nous  vous  sommes  reconnaissants  de  votre 
collaboration. 
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B.2  Sondage  3 

SONDAGE  A  L'INTENTION  DES  GESTIONNAIRES  POUR  EXPEDITION  09 
Confidentialite 

Nous  vous  remercions  de  I’interet  que  vous  manifestez  envers  ce  sondage  et  du  temps  que  vous  y 

consacrerez.  Vous  devriez  avoir  besoin  de _ minutes  tout  au  plus  pour  repondre  aux  questions. 

Veuillez  noter  que  toutes  les  reponses  demeureront  confidentielles,  et  que  I’identite  des  participants 
ne  sera  en  aucun  cas  devoilee  lorsque  les  resultats  seront  presentes.  Nous  vous  prions  de  lire  la 
section  suivante,  qui  porte  sur  le  consentement  des  participants. 

Consentement  eclaire 

J’accepte,  par  la  presente,  de  participer  au  sondage  sur  le  CRG  pour  RDDC. 

J’ai  bien  compris  que  le  sondage  vise  a  soutenir  le  role  que  joue  le  SMA(S  et  T)  dans  le  processus 
lie  au  CRG  du  CT,  de  meme  qu’a  simplifier  la  vie  des  gestionnaires  en  etablissant  clairement  leurs 
roles  et  leurs  responsabilites  ainsi  qu’en  leur  foumissant  un  cadre  qui  les  aidera  a  remplir  leurs 
fonctions. 

J’ai  bien  compris  que  ma  participation  est  purement  volontaire  et  que  je  suis  libre  de  cesser  de 
repondre  au  sondage  a  tout  moment. 

J’ai  bien  compris  que  les  donnees  brutes  obtenues  au  mo  yen  du  sondage  seront  codecs  de  fa^on  que 
ni  les  donnees  definitives  ni  les  rapports  qui  seront  produits  ne  refleteront  mon  identite. 

J’ai  bien  compris  que  tous  les  rapports  produits  a  la  suite  du  sondage,  qu’ils  soient  publics  ou 
internes,  seront  combines  afm  de  constituer  un  tout,  et  qu’en  aucune  fa^on  mon  identite  ne  sera 
revelee. 

□  J’accepte  ces  conditions  et  je  souhaite  participer  au  sondage. 

□  Je  n’accepte  pas  ces  conditions  et  je  ne  souhaite  pas  participer  au  sondage. 


DRDC  Atlantic  CR  2009-135 


263 


Section  A  -  TOUS  LES  NIVEAUX 


1 .  Quel  est  le  niveau  du  poste  que  vous  occupez  au  sein  de  RDDC? 

□  Niveau  2  (directeur  general  ou  chef  d’etat-major)  Go  to  Section  B 

□  Niveau  3  (directeur  general  adjoint,  directeur,  scientifique  en  chef,  chef  de  section, 
gestionnaire  ou  equivalent)  Go  to  Section  C 

□  Niveau  4  (superviseur,  expert  fonctionnel  ou  equivalent)  Go  to  Section  O 

□  Aucun  des  niveaux  susmentionnes  Go  to  Section  Q 
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Section  B  -  Niveau  2 


Les  questions  suivantes  out  trait  a  la  gestion  des  groupes  designes  vises  par  I’equite  en  niatiere 
d’emploi,  telle  que  dMnie  dans  la  Loi  sur  I’equite  en  matiere  d’emploi.  Les  quatre  groupes 
designes  sont  les  suivants  :  les  femmes,  les  Autochtones,  les  personnes  handicapees  et  les 
minorites  visibles. 

2.  A  votre  connaissance,  les  membres  des  groupes  designes  sont-ils  representes  de  maniere 
adequate  au  sein  des  groupes  de  base  des  postes  et  des  domaines  cles  de  votre  centre? 

□  Oui  □  Non  □  Jenesaispas 

3.  Est-ce  que  votre  centre  offre  des  possibilites  de  perfectionnement  professionnel  et  de  formation 
permettant  de  deceler  et  de  developper  les  talents  au  sein  des  groupes  designes? 

□  Oui  □  Non  □  Jenesaispas 

Go  to  Section  J 


DRDC  Atlantic  CR  2009-135 


265 


SECTION  C  -  Niveau  3 


4.  Dans  quel  centre  de  RDCC  travaillez-vous? 

□  Atlantique 

□  Valcartier 

□  Ottawa 

□  Toronto 

□  Suffield 

□  Centre  d’ analyse  et  de  recherche  operationnelle 

□  Directeur  general  -  Recherche  et  analyse  (Personnel  militaire) 

□  Centre  des  sciences  pour  la  securite 

□  Operations  S  et  T 

□  Services  generaux 

□  Chef  d’etat-major 

Aux  fins  du  present  sondage,  «  votre  unite  »  fait  reference  an  personnel  et  aux  ressources  dont 
vous  assurez  la  gestion  ou  la  supervision, 

5.  Etes-vous  responsable  de  la  gestion  d’un  budget  au  sein  de  votre  unite? 

□  Oui  go  to  Section  D  □  Non  go  to  Section  E 


266 


DRDC  Atlantic  CR  2009-135 


Section  D 


Les  questions  suivantes  ont  trait  a  la  gestion  budgetaire  an  sein  de  votre  unite.  Aux  fins  du 
present  sondage,  «  votre  unite  »  fait  reference  an  personnel  et  aux  ressources  dont  vous 
assurez  la  gestion  on  la  supervision. 

6.  Au  cours  du  dernier  exercice  financier,  votre  superviseur  a-t-il  impose  un  seuil  limite  pour  vos 
depenses  de  fin  d’annee  relativement  a  votre  budget  et  a  vos  previsions  de  depenses? 

□  Oui  (go  to  6a)  □  Non  (go  to  7) 

a.  Vos  depenses  de  fin  d’annee  correspondaient-elles  au  budget  ou  au  seuil  limite 
prevu  par  votre  superviseur? 

□  Oui  □  Non 

Proceed  to  question  7 

7.  Votre  unite  possede-t-elle  un  plan  strategique  d’immobilisations  qui  prend  en  compte  les 
besoins  a  long  terme? 

□  Oui  □  Non 

8.  Votre  unite  dispose-t-elle  d’un  plan  local  d’immobilisations  sous  forme  d’equipement  base  sur 
votre  allocation  budgetaire? 

□  Oui  □  Non 

9.  De  maniere  generate,  comment  evalueriez-vous  la  qualite  de  votre  relation  professionnelle 
avec  les  specialistes  financiers  de  votre  centre? 


□  Mauvaise  □  Moyenne  □  Bonne  □  Excellente 


S.O. 


Les  questions  suivantes  ont  trait  aux  activites  de  passation  des  marches  au  sein  de  votre  unite. 


10.  Le  nombre  de  specialistes  des  marches  de  votre  centre  vous  semble-t-il  adequat  par  rapport  au 
niveau  d’activite? 

□  Jamais  □  Rarement  □  Parfois  □  Souvent  □  Toujours 

11.  Combien  parmi  les  processus  internes  d’ appro visionnement  au  sein  de  votre  organisation  vous 
semblent  simples  d’ utilisation? 

□  Aucun  □  Quelques-uns  □  Laplupart  □  Tons 

12.  Est-ce  que  les  clients  qui  ont  a  faire  avec  le  processus  interne  d’ appro  visionnement  de  votre 
centre  sont  facilement  en  mesure  de  determiner  les  progres  realises? 
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□  Jamais 


□  Rarement 


□  Parfois 


□  Souvent 


□  Toujours 


13.  De  maniere  generale,  comment  evalueriez-vous  la  qualite  de  votre  relation  professionnelle 
avec  les  specialistes  des  marches  de  votre  centre? 

□  Mauvaise  □  Moyenne  □  Bonne  □  Excellente  □  S.O. 

Proceed  to  Section  E 
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Section  E  -  Gestion  de  I’information  et  des  ressources 


14.  Imaginons  qu’une  unite  ou  une  activite  dont  vous  etes  normalement  responsable  est  choisie 
pour  faire  I’objet  d’une  verification  menee  par  les  verificateurs  de  RDDC  ou  du  ministere  de  la 
Defense  nationale  (MDN).  Le  verificateur  pourrait-il  avoir  une  vision  globale  en  consultant 
uniquement  les  fichiers  et  d’autres  dossiers? 


□  Ova  go  to  Section  F 2  □  Sion  go  to  Section  F  1 
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Section  FI 


15.  Imaginons  qu’une  unite  ou  une  activite  dont  vous  etes  normalement  responsable  est  choisie 
pour  faire  I’objet  d’une  verification  interne.  Afin  d’obtenir  une  vision  globale  de  la  situation, 
dans  quelle  mesure  le  verificateur  devrait-il  vous  consulter  ou  consulter  vos  subordonnes  pour 
obtenir  des  renseignements  supplementaires? 

□  Une  quantite  importante  de  renseignements  supplementaires  serait  necessaire 

□  Une  quantite  moderee  de  renseignements  supplementaires  serait  necessaire 

□  Une  faible  quantite  de  renseignements  supplementaires  serait  necessaire 
Proceed  to  Section  F2 
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Section  F2 


16.  Quelle  est  la  demiere  fois  ou  vous,  ou  un  subordonne  responsable,  avez  inspecte  I’inventaire 
dont  vous  etes  responsable?  Si  vous  etes  gestionnaire  des  services  generaux,  cette  question 
porte  sur  I’inventaire  de  votre  unite  et  non  sur  I’inventaire  de  Torganisation  au  complet. 

□  Au  cours  des  6  demiers  mois 

□  Au  cours  des  7  a  12  demiers  mois 

□  Au  cours  des  1 3  a  24  demiers  mois 

□  II  y  a  plus  de  24  mois 

□  Jamais 

□  Je  ne  suis  pas  responsable  d’un  inventaire 

17.  Est-ce  que  vous,  ou  vos  subordonnes,  gerez  des  projets? 

□  Oui  Go  to  Section  G  □  Non  Go  to  Section  H 
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Section  G  -  Gestion  de  projet 


Les  questions  suivantes  portent  sur  la  gestion  de  projet,  A  titre  informatif,  le  Secretariat  du 
Conseil  du  Tresor  du  Canada  dMnit  la  gestion  de  projet  comme  « la  planification, 
I’organisation  et  la  surveillance  systematiques  des  ressources  attribuees  pour  atteindre 
certains  objectifs  et  resultats  des  projets  », 

18.  En  ce  qui  conceme  les  projets  geres  au  sein  de  votre  unite,  la  documentation  de  gestion  de 
projet  comprend-elle  une  matrice  des  responsabilites  ou  toute  autre  forme  d’ attribution  des 
responsabilites 

□  Oui  □  Non 

19.  Si  un  projet  gere  au  sein  de  votre  unite  ne  respecte  pas  les  etapes  prevues,  quels  sont  en  general 
les  elements  responsables  de  cet  echec? 


Tres  pen 

Pen 

Neutre 

Probable 

Tres 

probable 

probable 

probable 

Ressources 

1 

2 

3 

4 

5 

Calendrier 

1 

2 

3 

4 

5 

Objectifs 

1 

2 

3 

4 

5 

Qualite  des  etapes 

1 

2 

3 

4 

5 

20.  De  quelle  maniere  presentez-vous  aux  intervenants,  aux  clients  et  aux  partenaires  les  resultats 
provisoires  ou  fmaux  d’un  projet?  [choisissez  toutes  les  reponses  qui  s’appliquent] 


□ 

Exposes 

□ 

Rapport  standard 

□ 

Demonstrations 

□ 

Reunion 

□ 

Conferences 

□ 

Briefing  notes 

□ 

Articles  de  joumaux 

□ 

Other 
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2 1 .  Les  membres  du  personnel  publient-ils  des  articles  on  des  rapports  pour  tous  les  elements  de 
repartition  du  travail  des  projets? 

□  Jamais  □  Rarement  □  Parfois  □  Souvent  □  Toujours 

Proceed  to  Section  H 
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Section  H  -  Subordonnes 


Aux  fins  de  la  presente  section,  «  professionnels  des  sciences  et  technologies  (S  et  T) » 
comprend  les  groupes  professionnels  tels  que  les  ingenieurs,  les  chimistes  on  les  informaticiens 
qui  appuient  directement  les  programmes  des  S  et  T;  les  groupes  professionnels  tels  que  les 
technologues  n’en  font  pas  partie. 


22.  Avez-vous  des  subordonnes  directs? 

□  Oui,  et  certains  d’entre  eux  sont  des  scientifiques  de  la  Defense  ou  des 
professionnels  des  S  et  T  go  to  Section  I 

□  Oui,  mais  aucun  d’entre  eux  ne  sont  des  scientifiques  de  la  Defense  ou  des 
professionnels  des  S  et  T  go  to  Section  J 

□  Non  go  to  Section  K 
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Section  I  -  Subordonnes 


Aux  fins  de  la  presente  section,  «  professionnels  des  S  et  T  »  comprend  les  groupes 
professionnels  tels  que  les  ingenieurs,  les  chimistes  on  les  informaticiens  qui  appuient 
directement  le  programme  des  S  et  T;  les  groupes  professionnels  tels  que  les  technologues  n’en 
font  pas  partie.  Aux  fins  de  la  presente  question,  la  collaboration  suppose  une  interaction  en 
face-a-face  entre  vos  subordonnes  et  les  intervenants  externes,  et  pas  simplement  une 
participation  dans  le  cadre  de  projets  a  I’echelle  internationale. 

23.  Parmi  vos  subordonnes,  combien  d’ entre  eux  sont  des  scientifiques  de  la  Defense  ou  des 

professionnels  des  S  et  T?  Utilisez  le  niveau  de  dotation  autorise  plutot  qu’un  nombre  de  poste. 
NOMBRE 

a.  Parmi  vos  subordonnes  scientifiques  de  la  Defense  et  professionnels  des  S  et  T, 
combien  collaborent  avec  des  intervenants  du  MDN,  des  Forces  canadiennes  ou  de 
Securite  publique  Canada?  NOMBRE 

b.  Parmi  vos  subordonnes  scientifiques  de  la  Defense,  combien  collaborent  avec  des 
intervenants  canadiens  externes  (y  compris  les  fournisseurs)?  NOMBRE 

c.  Parmi  vos  subordonnes  scientifiques  de  la  Defense,  combien  collaborent  avec  des 
partenaires  etrangers  sous  I’egide  du  Programme  de  cooperation  technique,  de 
rOrganisation  du  traite  de  I’Atlantique  Nord  ou  d’autres  accords  intemationaux? 
NOMBRE 

d.  Parmi  vos  subordonnes  scientifiques  de  la  Defense  et  professionnels  des  S  et  T, 
combien  collaborent  avec  des  partenaires  de  I’exterieur  selon  le  regime  de  la 
remuneration  des  services  ou  le  principe  de  recouvrement  integral  des  couts? 

i.  Partenaires  canadiens  :  NOMBRE 


ii.  Partenaires  etrangers  :  NOMBRE 
Proceed  to  Section  J 
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Section  J  -  Niveaux  2  et  3 


Les  questions  suivantes  ont  trait  a  la  gestion  des  subordonnes  de  votre  unite.  Aux  fins  du 
present  sondage,  « votre  unite  »  fait  reference  an  personnel  et  aux  ressources  dont  vous 
assurez  la  gestion  on  la  supervision. 


24.  Les  plans  d’apprentissage  personnel  des  subordonnes  sont-ils  en  rapport  avec  les 
connaissances,  les  competences  et  les  capacites  appropriees  requises  pour  les  fonctions 
actuelles  ou  a  venir? 

□  Jamais  □  Rarement  □  Parfois  □  Souvent  □  Toujours 

25.  Avez-vous  discute  avec  vos  subordonnes  des  differents  moyens  possibles  de  transmettre  leurs 
connaissances  relatives  au  Ministere? 

□  Oui  □  Non 

26.  Encouragez-vous,  s’il  y  a  lieu,  les  subordonnes  a  travailler  en  equipe  et  a  foumir  une  formation 
polyvalente  pour  favoriser  le  transfert  des  connaissances  et  elargir  les  ensembles  de 
competences? 

□  Jamais  □  Rarement  □  Parfois  □  Souvent  □  Toujours 

27.  Avez-vous  examine  les  possibilites  qui  vous  permettraient  d’embaucher  un  nouvel  employe 
alors  que  le  titulaire  du  poste  demeure  toujours  au  sein  de  I’organisation  afm  de  faciliter  le 
transfert  des  connaissances? 

□  Oui  □  Non 


a.  Avez-vous  deja  embauche  un  nouvel  employe  alors  que  le  titulaire  du  poste  se 
trouvait  toujours  au  sein  de  T organisation  afin  de  faciliter  le  transfert  des 
connaissances? 

□  Oui  □  Non 


28. 


29. 


Votre  unite  possede-t-elle  assez  d’ equivalents  temps  plein  pour  remplir  son  mandat  avec 
succes? 

□  Oui  □  Non 

Les  postes  cles  sont-ils  pourvus  rapidement? 

□  Jamais  □  Rarement  □  Parfois  □  Souvent  □  Toujours 


276 


DRDC  Atlantic  CR  2009-135 


30.  Les  nouveaux  employes  affectes  aux  postes  cles  assument-ils  leur  role  de  maniere  efficace  et 
dans  un  delai  raisonnable? 

□  Jamais  □  Rarement  □  Parfois  □  Souvent  □  Toujours 

31.  Y  a-t-il,  au  sein  de  votre  unite,  des  employes  qualifies  prets  a  se  presenter  a  des  postes  cles  et 
dans  des  secteurs  cles? 

□  Oui  □  Non 

32.  Ciblez-vous  les  employes  dont  le  rendement  est  eleve  en  vue  de  la  planification  de  la  releve? 

□  Oui  □  Non 

Proceed  to  Section  K 
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Section  K  -  Niveaux  2  et  3 

Les  questions  suivantes  ont  trait  an  processus  d’examen  par  les  pairs. 

33.  Votre  unite  est-elle  soumise  a  Texamen  par  les  pairs  du  S  et  T? 

□  Oui  go  to  Section  L  □  Non  go  to  Section  N  □  Je  ne  sais  pas  go  to  Section  N 
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Section  L  -  Niveaux  2  et  3 


34.  A  quand  remonte  le  dernier  examen  par  les  pairs  pour  une  activite  des  S  et  T  organisee  au  sein 
de  votre  unite? 

□  Un  an 

□  Deux  ans 

□  Trois  ans 

□  Quatre  ans 

□  Cinq  ans 

□  Plus  de  cinq  ans 

□  Jamais 

□  Date  inconnue  Go  to  section  N 


Others  Proceed  to  Section  L2 
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Section  L2  -  Niveaux  2  et  3 


35.  Globalement,  quel  etait  le  resultat  de  votre  dernier  examen  par  les  pairs? 

□  Tres  □  Plutot  □  Neutre  □  Plutot  □  Tres  □  Je  ne  sais  pas 

negatif  negatif  positif  positif  Go  to  section  N 


Go  to  Section  M— 


SCREEN  RR  EAR  ********************************************* 
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Section  M  -  Niveaux  2  et  3 


36.  Globalement,  votre  dernier  examen  par  les  pairs  vous  a-t-il  permis  d’obtenir  une  retroaction 
utile? 

□  Non  □  Neutre  □  Oui 

37.  En  general,  quelle  est  1’ incidence  d’un  examen  par  les  pairs  sur  les  capacites  et  les 
competences  de  votre  unite? 

□  Aucune  incidence  □  Faible  incidence  □  Grande  incidence 

38.  En  general,  quel  est  le  type  d ’incidence  d’un  examen  par  les  pairs  sur  les  capacites  et  les 
competences  de  votre  unite? 

□  Negative  □  Neutre/Aucune  incidence  □  Positive 

39.  En  general,  quelle  est  V incidence  de  I’examen  par  les  pairs  sur  les  resultats  des  projets? 

□  Aucune  incidence  □  Faible  incidence  □  Grande  incidence 

40.  En  general,  quel  est  le  type  ddncidence  de  I’examen  par  les  pairs  sur  les  resultats  des  projets? 

□  Negative  □  Neutre/Aucune  incidence  □  Positive 


Proceed  to  Section  N 
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Section  N  -  Niveaux  2  et  3 


Les  questions  suivantes  ont  trait  aux  relations  de  travail  et  de  collaboration  que  vous 
entretenez  avec  des  partenaires,  des  clients  et  des  fournisseurs.  Aux  fins  du  present  sondage, 
on  entend  par  collaboration  le  fait,  pour  deux  parties  on  plus,  de  travailler  conjointement  afin 
d’atteindre  un  but  commun,  tout  en  partageant  les  responsabilites. 

4 1 .  Comment  evalueriez-vous  la  relation  de  travail  et  de  collaboration  avec  la  collectivite  des 
S  et  T  du  secteur  prive? 

□  Mauvaise  □  Moyenne  □  Bonne  □  Excellente  □  S.O. 

42.  Comment  evalueriez-vous  la  relation  de  travail  et  de  collaboration  avec  le  milieu  de 
I’enseignement? 

□  Mauvaise  □  Moyenne  □  Bonne  □  Excellente  □  S.O. 

43.  Avez-vous  ete  informe  des  resultats  du  dernier  sondage  sur  la  satisfaction  de  vos  clients  et  de 
VOS  partenaires  a  I’egard  de  votre  unite? 

□  Oui  □  Non  □  S.O. 

44.  A  quelle  frequence  recevez-vous  de  la  retroaction  directement  de  la  part  de  vos  clients  et  de  vos 
partenaires,  apres  la  mise  en  fonction  de  produits  de  S  et  T? 

□  Jamais  □  Rarement  □  Parfois  □  Souvent  □  Toujours  □  S.O. 

45.  Ea  retroaction  provenant  de  vos  clients  et  de  vos  partenaires  (y  compris  la  retroaction  directe  et 
le  sondage)  a-t-elle  une  incidence  sur  I’elaboration  ou  I’execution  de  projets  a  venir? 

□  Jamais  □  Rarement  □  Parfois  □  Souvent  □  Toujours  □  S.O 

46.  A  quelle  frequence  collaborez-vous  a  I’elaboration  de  politiques  ou  de  programmes  avec 
d’autres  intervenants? 

□  Jamais  □  Rarement  □  Parfois  □  Souvent  □  Toujours 

Level  2-  Go  to  Section  P  - Go  to  question  46a - 

Level  3-  Go  to  Section  O 
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a.  De  fafon  generale,  comment  evalueriez-vous  la  qualite  de  vos  collaborations? 

□  Mauvaise  □  Moyenne  □  Bonne  □  Excellente 

b.  Dans  quelle  mesure  votre  demiere  collaboration  vous  a-t-elle  permis  d’atteindre  les 
objectifs  vises  ou  de  satisfaire  a  vos  exigences? 

□  Aucun  objectif  atteint  ou  aucune  exigence  respectee 

□  Certains  objectifs  ont  ete  atteints  ou  certaines  exigences  ont  ete  respectees 

□  La  plupart  des  objectifs  ont  ete  atteints  ou  la  plupart  des  exigences  ont  ete 
respectees 

□  Tons  les  objectifs  ont  ete  atteints  ou  toutes  les  exigences  ont  ete  respectees 

Level  2-  Proceed  to  Section  P 
Level  3-  Proceed  to  Section  O 
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Section  O  -  Niveaux  3  et  4 

Les  questions  suivantes  ont  trait  a  I’entretien  de  I’equipement  utilise  an  sein  de  votre  unite. 

Aux  fins  du  present  sondage,  «  votre  unite  »  fait  reference  an  personnel  et  aux  ressources  dont 

vous  assurez  la  gestion  on  la  supervision. 

47.  Est-ce  qu’une  personne  de  votre  unite  est  affectee  offlciellement  a  I’entretien  quotidien  ou 
courant  de  Tequipement  utilise  dans  I’execution  de  vos  programmes  (p.  ex.,  le  calibrage)? 

□  Jamais  □  Rarement  □  Parfois  □  Souvent  □  Toujours  □  S.O. 

48.  Est-ce  qu’une  personne  de  votre  unite  est  affectee  offlciellement  a  I’entretien  quotidien  ou 
courant  de  I’equipement  relatif  a  E infrastructure  (p.  ex.,  le  remplacement  du  toner)? 

□  Jamais  □  Rarement  □  Parfois  □  Souvent  □  Toujours  □  S.O. 

49.  Lorsqu’un  equipement  doit  faire  Tobjet  de  travaux  d’entretien  ou  de  reparation  particuliers,  la 
procedure  visant  a  faire  une  demande  a  cet  effet  est-elle  facile  a  suivre? 

□  Jamais  □  Rarement  □  Parfois  □  Souvent  □  Toujours 

50.  Ees  demandes  de  reparation  d’equipement  sont-elles  traitees  rapidement  et  efficacement? 

□  Jamais  □  Rarement  □  Parfois  □  Souvent  □  Toujours 

Les  questions  suivantes  ont  trait  a  vos  experiences  an  sein  de  votre  unite  de  travail. 

5 1 .  Est-il  important  pour  votre  rendement  au  travail  de  vous  tenir  au  courant  des  idees,  des 
methodes  et  des  technologies  nouvelles  dans  votre  domaine? 

□  Pas  du  tout  important  □  Peu  important  □  Assez  important  □  Tres  important 

52.  Etes-vous  satisfait  des  ressources  ou  des  occasions  fournies  par  RDDC  afm  de  vous  tenir  au 
courant  des  idees,  des  methodes  et  des  technologies  nouvelles  dans  votre  domaine? 

□  Tres  insatisfait  □  Quelque  peu  □  Ni  satisfait,  □  Assez  satisfait  □  Tres  satisfait 
insatisfait  ni  insatifait 

53.  Votre  unite  a-t-elle  travaille  avec  des  etudiants  auparavant?  (Programme  federal  d’experience 
de  travail  etudiant,  Altemance  travail-etudes,  stages,  etc.) 

□  Oui  go  to  Section  02  □  Non  go  to  Section  P 
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Section  02  -  Niveaux  3  et  4 


54.  Comment  evalueriez-vous  1’ experience  de  votre  unite  relativement  au  travail  avec  des 
etudiants? 

□  Mauvaise  □  Moyenne  □  Bonne  □  Excellente 

Proceed  to  Section  P 
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Section  P  -  TOUS 


La  question  suivante  a  trait  aux  objectifs  strategiques  generaux,  Aux  fins  du  present  sondage, 
«  votre  unite  »  fait  reference  an  personnel  et  aux  ressources  dont  vous  assurez  la  gestion  on  la 
supervision. 

55.  En  general,  a  quelle  frequence  les  objectifs  generaux  suivants  representent-ils  un  facteur 
explicite  dans  le  travail  de  votre  unite? 


Jamais 

Rarement 

Parfois 

Souvent 

Toujours 

Perfectionnement  des  effectifs 

1 

2 

3 

4 

5 

Maintien  des  infrastructures 
scientifiques  et  techniques 
nationales  et  de  leur  capacite 

1 

2 

3 

4 

5 

Accroissement  de  P  ensemble 
de  connaissances 

1 

2 

3 

4 

5 

Commercialisation 

1 

2 

3 

4 

5 

Resultats  en  Recherche  et 
developpement  axes  sur  les 
besoins  d’une  mission 

1 

2 

3 

4 

5 

Les  questions  suivantes  ont  trait  a  la  gestion  de  I’information  dans  votre  centre. 

56.  Dans  votre  centre,  I’acces  aux  dossiers  du  personnel  est-il  restreint  au  personnel  du  Service  des 
ressources  humaines? 

□  Oui  □  Non  □  Je  ne  sais  pas 

57.  Dans  votre  centre,  la  correspondance  classifiee  fait-elle  I’objet  d’un  suivi?  (p.  ex.,  identificateur 
d’utilisateur  unique,  registre  de  documents,  etc.) 

□  Oui  □  Non  □  Je  ne  sais  pas 

58.  Dans  votre  centre,  la  correspondance  classifiee  est-classee  dans  un  endroit  sur? 
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□  Oui 


□  Non 


Les  questions  suivantes  out  trait  an  Programme  de  mentorat  de  RDDC 


59.  Avez-vous  participe  au  Programme  de  mentorat  de  RDDC? 


□  Oui  go  to  59a  □  Non  go  to  59c 


SCREEN  BREAK* ******************************************** 


a.  Quel  etait  votre  role?  Cochez  toutes  les  reponses  qui  s’appliquent. 

□  Mentor 

□  Mentore 

□  Membre  de  I’equipe  du  Programme  de  mentorat 

b.  Comment  evalueriez-vous  votre  satisfaction  quant  au  Programme  de  mentorat  de 
RDDC? 


□  Mauvaise  □  Moyenne  □  Bonne 


□  Excellente 


Go  to  question  60 

Hc****************§(;;;pj|;p;]q  gj^g^j^********************************************* 

c.  Si  vous  avez  repondu  non,  quelle  impression  avez-vous  du  Programme  de  mentorat  de 
RDDC? 


□  Positive  □  Negative  □  Pas  d’ opinion  □  Je  ne  connais  pas  ce 

programme 

Go  to  question  60 

Hc****************§(;;;ggp;]q  gpg^j^********************************************* 
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60.  Etes-vous  interesse  a  participer  au  Programme  de  mentorat  de  RDDC  a  I’avenir? 


□  Oui  go  to  60a  □  Non  go  to  61  □  Incertain  go  to  61 

a.  A  quel  titre  seriez-vous  interesse  a  participer  au  Programme  de  mentorat? 

□  Mentor 

□  Mentore 

□  Membre  de  I’equipe  du  Programme  de  mentorat 


Proceed  to  question  61 


61.  Veuillez  noter  ci-dessous  vos  commmentaires  sur  le  Programme  de  mentorat  de  RDDC  : 
(textbox) 


Proceed  to  Section  Q 
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Section  Q 


Nous  avons  fait  le  tour  des  questions  que  nous  voulions  vous  poser  aujourd’hui.  Si  vous  avez 
d’autres  comnientaires  a  forniuler  au  sujet  de  ce  sondage,  veuillez  les  inscrire  ici : 

(text  box) 

Merci  d’avoir  repondu  a  ce  sondage.  Votre  participation  est  appreciee. 
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ADM 

ADM(S&T) 

ARP 

ATI 

BERD 

CANDID 

CORA 

COSO 

CPME 

CPSA 

CR 

CRG 

CRP 

CRS 

CRTl 

CSPS 

CT 

DAOD 

DGMPRA 

DGRDCS 

DGSTO 

DIRP 

DM 

DND 

DPR 

DRDC 

DRDKIM 

DRENet 

DSAR 

DSTEA 

DSTER 

DWAN/DIN 

EAP 

EE 

FAA 

FMAS 

GERD 

HERD 

HR 

HRM 

HRO 

ICMS 

IRM 


Assistant  Deputy  Minister 

Assistant  Deputy  Minister  (Science  and  Technology) 

Applied  Research  Program 
Access  to  Information 

Business  Expenditure  on  Research  and  Development 

Canadian  Defence  Information  Database 

Centre  for  Operational  Research  and  Analysis 

Committee  of  Senior  Officials 

Collaborative  Program  Management  Environment 

Canada  Public  Service  Agency 

Contract  Report 

Cadre  de  Responsabilisation 

Corporate  Risk  Profile 

Chief  Review  Services 

CBRN  (Chemical,  Radiological,  Nuclear  and  Biological)  Research  and 

Technology  Initiative 

Canada  School  of  Public  Service 

Conseil  du  Tresor 

Defence  Administrative  Orders  and  Directives 

Director  General  Military  Personnel  Research  and  Analysis 

Director  General  R&D  Corporate  Services 

Director  General  Science  and  Technology  Operations 

Defence  Industrial  Research  Program 

Deputy  Minister 

Department  of  National  Defence 

Departmental  Performance  Report 

Defence  Research  and  Development  Canada 

Director  R&D  Knowledge  and  Information  Management 

Defence  Research  Network 

Departmental  Staffing  Accountability  Report 

Director  Science  and  Technology  Enterprise  Affairs 

Director  Science  and  Technology  External  Relations 

Defence  Wide  Area  Network/Defence  IntraNet 

Employee  Assistance  Program 

Employment  Equity 

Financial  Administration  Act 

Financial  Management  System 

Gross  Expenditure  on  Research  and  Development 

Higher  Education  Expenditure  on  Research  and  Development 

Human  Resources 

Human  Resource  Management 

High  Reliability  Organization  Theory 

Informal  conflict  management  system 

Integrated  Risk  Management 
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MAF 
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MDN 

MRRS 

NDHQ 

OL 

OPl 

PAA 

PD 

PER 

PGA 

PEP 

PMA 

PRR 

PS 

PSC 

PSES 

PSMA 

R&D 

RDDC 

RDEC 

RM 

RPP 

RTA 

S&T 

SCT 

SMAF 

SOP 

SSRRB 

TAG 

TB 

TBS 

TDP 

TOS 

WEDC 


Eabour  Management  Coordination  Committee 
Eabour  Management  Relations  Committee 
Management  Accountability  Framework 
Modem  Comptrollership  Initiative 
Ministere  de  la  defense  nationale 
Management,  Resources,  and  Results  Structure 
National  Defence  Headquarters 
Official  Eanguages 
Office  of  Primary  Interest 
Program  Activity  Architecture 
Professional  Development 

Performance  Evaluation  Report  (for  the  Defence  Scientist  occupational  group) 
Partner  Group  Agreement  (a  service  level  agreement  on  planned  S&T  activities) 
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